Gurobi hosted a webinar by Irv and Rob to discuss innovative optimization work with Birchbox, the trailblazing e-commerce business. They were joined by Birchbox VP of Global Business Technology David Bendes, who leads subscription technology and operations, data and business intelligence, and platform product management, focusing on creating data‑driven experiences and the technologies that power them. We summarized the technical aspects of this work in a previous post; following are lightly edited excerpts from the webinar.
Irv Lustig: Our topic today is “Formulation Matters: Reciprocating Integer Programming for Birchbox Product Assortment.” The key message is that your formulation of the problem for a MIP solver makes a difference in getting the best performance.
David Bendes: At Birchbox, we’re trying to help casual beauty consumers find and discover the products that work for them in their routines. We launched in 2010 and have added wisdom to approach things in different ways. The experience that Birchbox wants to give our customers is governed by how we operate the “black box” that allows us to build custom assortments for different people, give them all types of customization, and allow them to choose different samples. We take our subscriber profiles—their history, activity, what samples they are liking, products they are buying, things have they gotten in the past and how they reacted—along with different product and vendor attributes we have. We use that to figure out the optimal assortment of samples into boxes.
We’ve been using a MIP model with Gurobi to optimize that process. We started running into a few different issues: when we wanted to innovate on our customer experience, we found that the model was actually a blocker. It had worked for quite a long time. Our customer base grew and we wanted to expand our offering, experiment with different numbers of products in the box, experiment with added customization, and really drive an innovative experience, so we added those constraints but then the run‑time of the model increased from hours to multiple days. When that happens, operations can go up against all types of strange production deadlines, and we really get stuck, forced to battle our operation tooling. We had business ideas we wanted to unlock that our model wasn’t able to handle. That's when we reached out to Princeton Consultants to see how we could improve the performance of our existing model to support the innovative experiences we wanted to bring forward.
Irv: Let’s talk about some of the lessons from this exercise. One is that formulation matters. While the solvers have gotten faster, you’ve also seen that business problems are getting harder and expectations sometimes can be unrealistic. Even with all the great performance improvements that Gurobi has had over the last several years, a bad formulation can doom your model. It’s often really easy to formulate a model but it can be really difficult—and there's an art and a science to this—to build a model that handles the scale and complexities of different business operations, provides solutions in a reasonable amount of time, and gives you a robust performance over a wide variety of data sets. The second lesson is that it’s not always important to get optimality. Sometimes you have to understand the objectives in the context of the business problem. There might be multiple objectives conflicting. In this problem, two of the three objectives were really about whether things were feasible: can we get every subscriber a box, and can we do it in the required number of boxes? It was important to know that as soon as possible. The third objective is about whether we can meet the soft constraint of hitting the product minimums. Here is a set of additional lessons:
- When you’re doing things like this and developing algorithms, it's important to have a representative data set for continual testing. As you change your model, things might help some data sets but not others. In our practice, we collect a robust data set to make sure that the changes were worked on in a robust way.
- Don't be afraid to challenge the status quo when it comes to the business problem. Sometimes you have to ask why are they doing different things and why things are acting that way. By coming in with a new perspective, you can basically allow the team to consider new angles and "think outside the Birchbox" if you will.
- Finally, leverage the power of the Gurobi MIP. In a lot of column-generation schemes, people tend to build heuristics to generate new columns and patterns. As you can see from this case, MIP does this quite well.
David: Let’s get back to the business side because this all has to drive a business case, right? The point was to build an innovative customer experience to unlock different things where we could go and test new features really quickly, and give more customization to our customers. This model has really given us the ability to do that, going from a run‑time of several days to only 15 minutes—life‑changing for me! Now we can test, for example, "Oh, you want to add a sample in," or if something happens with our supply chain, especially with COVID going on. If we don't have a sample that we thought we would have, now we can make a new assortment and test that in minutes. The impact on our business is so dramatic. We essentially have 4-8 days back to focus on having more effective production. We can focus on working with our target vendors that are making more effective product cards that leverage all the different types of samples, as opposed to having to rush this out the door and really be hunting for a solve. We can maximize the amount of customization we give to our customers. We can do a lot with that extra time. From a business perspective, and from a customer experience perspective, it's been game‑changing for us.
Irv: Dave, you said you have adapted during COVID.
David: Mainly because our supply chains are up in the air. Some vendors may have slight production delays, in which case they can't get samples to us in time to actually put it in, let’s say, the June box. If we don’t have that sample in the month of May, it’s really hard to build it into a June box. This model allows us to say, “Well, what else can we take? What other samples can we bring in?” It's almost a hedge against uncertainty, being able to have something that works so much faster.
Regarding project ROI, there were a few KPI guides we were looking at. One was binary: does this model allow us to unlock the things that we want to do or not? Additionally, we look at our production costs and look at how much more customization we can give to our customers. From a business perspective, that is a customer choosing to engage in a customization feature and saying, "Give me this sample in my next box." From an execution perspective, we take a certain percentage of random people and examine if they must get a certain sample, can we build an assortment subject to that constraint? Being able to increase that percentage is our way of increasing that product. Before, when we were running against a production deadline and had multiple days of run-time, it was much harder to maximize that value. Those were some of our secondary KPIs.
Contact Princeton Consultants to discuss this work further and other optimization opportunities.