Leaders of Remsoft, which provides optimized planning and scheduling for land-based assets, sought to eliminate performance issues that were impacting in-house and client-side users of their software. Attempts to uncover the root causes, which were inhibiting solutions for a growing category of business problems, had been unsuccessful for several years. Ultimately, their identification and correction required a formal quality assurance evaluation by an outside party with expertise in modeling, interior point algorithms, numerical computing, solvers, and deployment of optimization systems.
“For a quality assurance assessment based on optimization systems and computational mathematics, I don’t believe you could find a blend of talent and service better than Princeton Consultants. They combined the necessary technical expertise with methodical interviews of our leaders to promptly understand our models and practices, get to the heart of performance problems, and present the necessary fixes. They further recommended changes that have helped us advance our development and services for our clients around the world.”
Andrea Feunekes
Co-Founder, Chief Executive Officer
Remsoft
Challenges for a Proprietary Platform that Generates Many Types of Models
Founded over 25 years ago, with clients worldwide, Remsoft sells and maintains software based on its own modeling platform for stating optimization problems where it is desired to efficiently manage assets over time. The platform relies on a proprietary modeling language to represent assets, transformations of those assets over time, and outputs due to those transformations, along with the associated data—combined with objective functions and constraints on the outputs—that generate linear programs for an optimization solver. Remsoft also created a scripting language that is used by its consultants to extract data from spreadsheets and databases, and then transform that data into the appropriate format needed by the modeling platform.
Historically, the software has supported many linear programming solvers, including solvers for only small problems. Recently, the platform was extended to support the development of Sales and Operations Planning (S&OP) models, which generated mixed integer programs (MIPs) that can be difficult to solve. Developers encountered inadequate solution times of their formulations of MIPs, especially for larger problems with many assets and time periods.
Many of Remsoft’s clients, who are proficient in the modeling platform and write their own models, have demanded increased asset and temporal granularity. The data requirements of those planning problems led the company to offer professional services to assist in building models and integrating data. The most challenging optimization problems generated by the platform are often built by Remsoft’s consultants.
It was always difficult to tune an optimization solver to achieve good performance for the company’s software, mainly because the proprietary modeling platform allows users to generate a wide variety of models, rather than one class of models. There was no onesize-fits-all for appropriate solver settings and modeling techniques. This was especially true in the case of the models that generate MIPs because different business problems that leverage mixed integer programming modeling techniques may require specific formulations as well as problem-specific solver tuning.
Conducting Advanced Analytics Quality Assurance
Remsoft’s leaders engaged Princeton Consultants for its Advanced Analytics Model Review and Validation services, which are based on a proven methodology for developing and deploying optimization-based software systems into production.
The initial activities included interviews of key company personnel to understand the business problem and context, and to determine the current structure of the different models, including decision variables, constraints and objective functions.
Princeton Consultants reviewed documentation to understand the modeling platform and the data sources. Data sets were reviewed to understand how the data and modeling platform mapped to a model’s implementation, looking for differences between the understanding of the mathematical model and the actual implementation. Finally, Princeton Consultants analyzed and experimented with several key optimization instances from a variety of Remsoft’s clients considered to be particularly challenging.
Recommendations
Early in the model review, it was discovered that the use of single precision in the matrix generator was causing numerical difficulties for optimization solvers. Changing the matrix generator to use double precision arithmetic was recommended and promptly enacted.
It was found that certain constraints were computed to multiple significant digits after the decimal point, which was too precise for the underlying entity. Carefully rounding computations was encouraged so that values input to the optimizer were reflective of the appropriate accuracy.
Further recommendations to improve platform performance for potentially all users included:
- Add unit specifications so that the matrix generator could round values based on an understanding of the semantic meaning of the underlying unit of measurement
- Rename decision variables and constraints to facilitate debugging and further analysis
- Avoid artificial perturbations by users who were trying to correct for issues that were occurring due to the use of single precision and the lack of rounding.
For a general class of models generated by the platform, Princeton Consultants recommended best practices, such as the use of hierarchical objectives when multiple objectives are present, and the consideration of underlying units of the constraints when determining appropriate penalties. Princeton Consultants suggested settings for Remsoft’s main solver partner, Gurobi, to be distributed to users.
Additionally, Princeton Consultants evaluated three critical models and suggested a series of new modeling techniques and changes in formulations.
The Path Forward
Princeton Consultants and Remsoft’s leaders agreed on a set of immediate activities, as well as the analysis of additional models. The formal, third-party model review—which lasted only two weeks—uncovered issues and improvements that had eluded in-house experts for several years. Solution times on some problems reduced from over 8 hours to 10 minutes. The recommended actions will promptly increase the reliability and robustness of Remsoft’s software, and enable ongoing improvements to solution development.
About Remsoft
Remsoft, www.remsoft.com, is a global leader in optimized planning and scheduling software solutions for forestry and other land-based and infrastructure assets such as forests, roads, farms, energy resources and more. For more than 25 years, Remsoft has helped organizations develop business and environmental sustainability by assisting them in making decisions about how to acquire, manage, maintain and sell their assets.
To learn more about Princeton Consultants' implementation verification methodology, visit this post. We welcome conversations about model review and validation--email us to set up a call.