Princeton Optimization

Menu

Have Any Questions?

+1 800 34 678 900

Case Studies

Assets to Optimize

People

Employees, vendors, customers, shoppers

Materials

Raw materials, components, finished goods

Containers

Storage or shipping containers, shelf space

Vehicles/Machinery

Trucks, planes, trains, ships, manufacturing equipment

LandLand/Facilities

Farms, distribution centers, transit points

Finances/Investments

Capital funds, investment portfolios

Digital & Intangibles

Web space, TV/radio time, intellectual property

Optimization Decision Types

Assignment

Who gets what?

Production/Acquisition

How much to make/get/have?

Scheduling

When? In what order?

Network Routing

Where? Which path?

Pricing

At what price?

Policy Creation

What should the rules be?

Complexity Factors

Big Data

Extraordinarily large data sets or numbers of variables.

Noisy Data

Missing, conflicting or erroneous data.

Stochastic

Not certain, subject to probability.

Competitive Gaming

Competitors are acting on our actions.

Human In The Loop

A human decision maker is in the “loop” for each decision.

Black Box

The optimization has no human in the loop for each decision.

Static/Dynamic

The problem is solved up front, but must be resolved with constantly updating information.

Real Time

The optimization must give split-second responses.

Search

account

cart

  • Main menu

People

Employees, vendors, customers, shoppers

For many businesses, core processes involve decisions involving the key asset of their workforce, as well as contracted workers. 

In optimization applications, people are usually governed by multiple overlapping sets of costs and rules–these represent both challenges and opportunities.  The challenges are to find the best solutions that follow all the rules.  The opportunities are to try different combinations in different circumstances that follow the rules and maximize the formal objective function, which is typically the highest productivity and lowest cost.

At Princeton Consultants we group these costs and rules as follows: physical, regulatory, contractual, marketplace, policy and practical.

  • Regulatory constraints include laws governing overtime, maximum work hours, and other factors — e.g. the U.S. Department of Transportation specifies maximum continuous and weekly hours driving for commercial truck drivers.  Sometimes multiple jurisdictions apply:  for example, at the airline a pilot flying from the U.S. to Europe  moves from rules of the Federal Aviation Administration to those of the European Aviation Safety Agency. For the biotech manufacturer, where personnel are specialized and highly paid, leaders can answer strategic questions such as “How do I optimize operator shifts to maximize throughput and minimize overtime?”
  • Contractual terms include employment and labor agreements.  For the service network company, executive management employed optimization to generate and evaluate recommendations for maximum revenue and profit strategic options, given a portfolio of different business units, each with different work rules.
  • Marketplace and Policy rules and constraints include internal rules on how assignments are made – whether they are tasks, accounts or territories. The 2020 U.S. Census will be conducted by more than 400,000 temporary workers called “enumerators,” who will work for several weeks with variable daily and weekly schedules. At the e-commerce trailblazer, operations executives were forced to painstakingly manage and monitor the model because it too often could not find adequate solutions and, when it could, the average run time of 30-50 hours jeopardized production deadlines.
  • Practical rules capture important real-world observations that relate to the fact that people are individuals and not machines.  People who have to navigate a rule – on land or in the air or sea – find that their efficiency improves as they run the same route.  Consequently, the optimization model should tend to favor assigning people to repeating routes.  On the other hand, merely assigning the same route to the same pilot or driver might not always be the most efficient. For the regional agribusiness, workers at the same level and even pay attain different levels of productivity at different tasks. As a result, the optimizer can experiment with People assignments to gauge task / person efficiencies to find the best pairings.

Materials

Raw materials, components, finished goods

Optimization is classically used to make the highest profit choices in supply chain applications at all levels.

In optimization applications, materials are usually governed by multiple overlapping sets of rules and costs. These rules represent both challenges and opportunities. The challenge is to find the best solutions that follow all the rules. The opportunities are to try different combinations in different circumstances that follow the rules and maximize the formal “objective function” – typically: highest service at lowest cost.

At Princeton Consultants, we group these rules/costs as follows: physical, regulatory, contractual, marketplace, policy, and practical.

  • Physical: For the regional agribusiness, the crops have a high rate of perishability, so the optimization and reduction of the time from harvest to milling had a major impact on food quality and the price the company was able to obtain. For the biotech manufacturer, the product is urgent, personalized cell therapy that is highly perishable. Simulation identifies resource bottlenecks and best use of capital improvements; the scheduling optimization solution in part balances resource utilization.
  • Regulatory: For the freight railroad, different classes of shipments had different legal characteristics. For instance, Hazardous Materials (Hazmat) have higher costs and limit the vendors and locations they can ship through.
  • Contractual: for the business publication, different customers buy guaranteed advertising plans. The company’s ability to promise and deliver on highly flexible and tailored plans with optimization has been key to attracting and retaining top accounts at margins higher than its competitors.
  • Marketplace, Policy, and Practical: The printing company found that with optimization it could improve on the postal rates its magazine customers paid, putting money in its pockets and standing out in an otherwise highly commoditized industry. At the e-commerce trailblazer, the optimization model places components/products into containers/boxes and reduces the required number of box configurations to meet subscriber needs; the transformed performance allows the team to tweak inputs and re-run the model—and therefore evaluate different parameters, levels of subscriber aggregation, definitions of a “good” box, and even optimize on box value.

Containers

Storage or shipping containers, shelf space

Containers merit their own Asset class because of their value and unique characteristics. Businesses that sell capacity are often best modeled using Container logic, whether the container is a classic “box,” such as a shipping container, or another kind of valuable capacity, such as advertising space in a business publication.

In optimization applications, containers are usually governed by multiple overlapping sets of rules and costs. These rules represent both challenges and opportunities. The challenge is to find the best solutions that follow all of the rules. The opportunities are to try different combinations in different circumstances that follow the rules and maximize the formal objective function, which is typically greatest asset utilization and minimized repositioning.

At Princeton Consultants we group these rules and costs as follows: physical, regulatory, contractual, marketplace, policy, and practical.

  • Physical and Regulatory: The capacity of a container may vary depending on simultaneous constraints. For instance, in freight transportation such as the freight railroad, weight and cube maximums govern capacity. For the airline, passengers book with multiple sized parties, and the plane’s capacity is measured by seats.
  • Contractual: Often, capacity is discounted if the buyer guarantees usage. For the service network company, high volume users are given different rates and service guarantees than are smaller accounts.
  • Marketplace and Policy: At the e-commerce trailblazer, the optimization solution must be formulated to deliver product groupings and box creations deemed “ideal” by the company’s proprietary criteria. For the business publication, optimizing advertising revenue means juggling multiple layouts and configurations to achieve for each customer the space with the most impact and readership/viewership.
  • Practical: The best optimization gives a business an edge by enforcing practical rules. For the pharmaceutical company, reducing the number of “change outs” in the test trays that held the sample tubes had the practical impact of reducing errors by the lab staff.

Vehicles and Machinery

Trucks, planes, trains, ships, manufacturing equipment

Optimization is classically used to make the highest profit choices in multiple decision types for vehicles and machinery, including: assignment, production/acquisition, scheduling, network routing, pricing, and policy creation.

In optimization applications, these decisions are usually governed by multiple overlapping sets of rules and costs. These rules represent both challenges and opportunities.

The challenges are to find the best solutions that follow all of the rules. The opportunities are to try different combinations in different circumstances that follow the rules and maximize the formal objective function: typically, greatest asset utilization and maximum profit.

At Princeton Consultants, we group these rules/costs as follows: physical, regulatory, contractual, marketplace, policy, and practical.

  • Physical: For the regional agribusiness, we helped management achieve significantly higher utilization of rail engines and cars by modeling the physical track’s costs, constraints, and decision points. For the biotech manufacturer, where some of the machines are very expensive, simulation enables evaluation of the impact of changes in process, resources, and sequence on throughput, timeline reliability, turnaround time, product quality and cost; and a production scheduling optimization solution maximizes throughput, minimizes cycle time, and balances resource utilization.
  • Regulatory: Vehicles and machinery often are subject to regulations that can be complex, for instance the airline, in selecting a plane and airport, must respect minimum runway lengths for different types of planes in different weather conditions and weights.
  • Contractual: For magazines, the printing company must closely coordinate its binding to be at the exact day and sometimes time of day with the printing presses and the outbound transportation.
  • Marketplace: For the pharmaceutical company, demand for individual allergy tests varied by season, therefore allowing us to help its management optimize by configuring the machines in advance to the most efficient setup for forecasted demand.
  • Policy: For the business publication, it is company policy not to run advertisements on the same pages as news stories that concern that company, a competitor, or its industry.
  • Practical: In the service network company, changing configurations too often would drive down the efficiency of the operations staff, so Princeton Consultants added a cost into the optimization that would be unafraid to make network routing changes, while at the same time reflecting true and nuanced costs.

Land and Facilities

Farms, distribution centers, transit points

Optimization is classically used to make the most profitable choices in multiple decision types for vehicles and machinery, including: assignment, production/acquisition, scheduling, network routing, pricing, and policy creation.

In optimization applications, these decisions about Land and Facilities are usually governed by multiple overlapping sets of rules and costs. These rules represent both challenges and opportunities. The challenges are to find the best solutions that follow all the rules. The opportunities are to try different combinations in different circumstances that follow the rules and maximize the formal objective function, which is typically greatest asset utilization and minimum overtime and transportation costs–and therefore maximum profit.

At Princeton Consultants, we group these rules/costs as follows: physical, regulatory, contractual, marketplace, policy, and practical.

  • Physical: train capacity for the freight railroad terminals; distance from the fields to the mills for the regional agribusiness
  • Regulatory: landing rules for the airline by country and by airport
  • Contractual: union work rules for the service network company
  • Marketplace: demand varies by time of year and crop varietal for the regional agribusiness
  • Policy: crop rotation by field for the regional agribusiness
  • Practical: congestion avoidance based on time of day for the freight railroad’s discretionary movements

Finances and Investments

Capital funds, investment portfolios

Finance and Investments merit their own Asset class because of their value and their unique characteristics.

For strategic investment analysis, optimization can elevate traditional spreadsheet-driven models and therefore provide a quantitative edge.

  • For the service network company, we helped executive management evaluate strategic investments in various business units, as well as potential spin-offs and acquisition targets. To evaluate these choices, we used optimizing-simulation to evaluate the network effects of these choices (simulation) and to recommend the choices with the highest returns (optimization).

Perhaps no better example of the power of optimization for finance is under conditions of Big Data.

  • The high frequency hedge fund makes real-time, black box trading decisions among thousands of equities.

Digital and Intangibles

Web space, TV/radio time, intellectual property

Beyond finance and investments, non-physical assets cover a wide range of categories that include contractual assets or opportunities.

  • The equipment management company uses optimization to balance the contractual entitlements of each of its major customers to a shared pool of assets.
  • The printing company successfully uses optimization to minimize postal delivery costs by adroitly binding in such a way to maximally benefit from postal service incentives to perform tasks such as bundling and sorting by carrier route.

Sometimes digital assets are combined with physical assets:

  • The business publication sells coordinated advertising programs in both its print and its online versions.

Some businesses find the optimization system itself becomes a valuable digital asset:

  • The freight railroad attracts brokers by providing its online optimization system to them via a web portal. This visibility allows brokers to instantly find the best possible routes and costs among thousands or millions of choices.

Assignment

Who gets what?

Many business processes involve assignment: matching scarce resources to jobs, tasks, regions, or choosing to hold them in reserve.

Without optimization, businesses use rules that are of the form: “Given conditions X, Y, and Z, assign A to B.” Sometimes these rules are programmed; sometimes they are in people’s heads. Neither is optimization. In virtually all cases, optimization can provide not only a better assignment, but a significantly better set of assignments.

  • For the airline, crew are assigned to a plane and crew/planes are assigned to a series of pickup and delivery stops that form “a flight line.” As with most transportation carriers, the airline also decides which flights to assign to its own fleet and which flights to assign to its vendors. Key to the assignment is minimizing empty air miles and wasted jet fuel, as well as maximizing crew utilization and jet utilization.
  • For the business publication, advertisements are assigned to sections, and blank pages are assigned to these sections to carry the ads. Key to the assignment is providing customers with their placements of choice. This requires juggling the many combinations that arise when a customer expresses multiple choices to accommodate as many customers as possible.
  • For the regional agribusiness, assignment in a dynamic level includes which railcars should be used to transport which waiting crop loads. Key to the optimization is maximizing asset utilization while minimizing the time between the time the crops are harvested and milled.
  • For the pharmaceutical company, assigning blood serum samples to different tests and test equipment is used to maximize diagnostic power and minimize costs and wasted slots in the machines.
  • For the printing company, pockets in the complex binding line are assigned to batches of similar pages to create magazines in a way that maximizes the throughput of the binding line and minimizes costly change-outs.
  • For the biotech manufacturer, high-value machines and personnel are assigned through a production scheduling optimization solution that maximizes throughput, minimizes cycle time, minimizes operator overtime, and balances resource utilization.
  • For the e-commerce trailblazer, a custom optimization model groups products and assigns subscribers according to proprietary objectives and constraints tied to subscriber profiles, history, and activity, product and vendor attributes.
  • For the 2020 U.S. Census, automation enables significant changes as to how cases are assigned and how the field staff is supervised. By making it easier for supervisors to monitor and manage their workers, the ratio of workers to supervisor can be increased, reducing the number of supervisors required.

Production and Acquisition

How much to make/get/have?

Production/acquisition decisions usually involve what and how much to either manufacture or to purchase.

Prior to optimization, production/acquisition decisions are often made simply according to rules, such as “When the amount is under X%, then reorder.” Optimization can provide smarter, dynamic decisions that consider many more features than simple rules, achieving better results.

  • For the high frequency hedge fund, positions in financial securities are created, increased or decreased to optimize return by maximizing forecasted return while minimizing risk
  • For the business publication, different sections are enlarged or shrunk to maximize advertising revenue
  • For the airline, the optimization is run 1, 2 and 3 days in advance to help the business determine how much vendor capacity will be needed ahead of time, and acquire/negotiate the best prices
  • For the biotech manufacturer, optimizing simulation enables executive to test the scalability of existing processes, identify resource bottlenecks and best use of capital improvements, and assess tradeoffs between different objectives such as cost vs. throughput and turnaround time vs. quality.
  • For the e-commerce trailblazer, the custom optimization solution delivers product groupings and box creations deemed “ideal” by the company’s proprietary criteria. Additionally, the solution can accommodate meaningful business changes and scale as the service grows. In 2015, the company had over a million subscribers and more than 800 brand partners.

Scheduling

When? In what order?

Scheduling involves deciding on the order and time for a series of actions.

Scheduling may seem like a simple problem at first, but whenever the decision includes the order to do things, the possibilities become factorial — i.e., fantastically large. For instance, scheduling the order in which to perform 10 tasks entails over 3 million possible combinations (10! Or 3,628,800). Simply adding 3 more tasks increases the number of possible combinations to over 87 billion. (This is discussed in The Optimization Edge: page 55).

Because of this fantastic number of combinations, in the absence of optimization, organizations tend to create simple rules for making decisions, such as “First In, First Assigned” (affectionately referred to by optimizers as “First Pig to the Trough”) or “Assign the Biggest First” or “Assign the Highest Profit First”, or other similar approaches. In many organizations, savvy insiders know that the scheduling process essentially consists of copying the last schedule and simply editing in changes in supply and demand.

One of optimization’s great contributions to a business comes from its ability to evaluate and score billions of combinations in seconds – something no human, even aided with a “rules engine” can approach.

  • For the airline, optimized schedules are created for revenue tasks (transporting customers) as well as non-revenue tasks (repositioning planes and crew). The resulting schedule can perform the “double hat trick” of dramatically reducing costs while improving customer service. In a nutshell: putting together the right schedule reduces asset waste and frees the assets to then be in the right place at the right time for customers.
  • For the high frequency hedge fund, schedules are created to govern which financial positions are acquired and unwound at what times and in what order to maximize return and to comply with balance and liquidity constraints.
  • For the pharmaceutical company, blood samples are scheduled to test machines in a way that minimizes waiting for diagnostic results for the most critical cases, while maximizing overall lab throughput.
  • For the biotech manufacturer, the custom production scheduling optimization solution maximizes throughput, minimizes cycle time, minimizes operator overtime, and balances resource utilization.
  • For the 2020 U.S. Census, the Non-Response Follow Up (NRFU) operation will be conducted by over 400,000 temporary fieldworkers called “enumerators,” who will work for several weeks with variable daily and weekly schedules.

Network Routing

Where? Which path?

Some of the most sophisticated decision-making involves the consideration and management of movement through networks.

At Princeton Consultants, we consider networks as a series of paths through four dimensions: Space, Time, Activities and Resources (S.T.A.R.).

  • For the freight railroad, customer freight routing decisions include identifying the closest terminals (Space), drive time, gate cutoffs and train schedules (Time), determining which vendors can provide the required services for each load (Activities) and equipment capacity (Resources). Key to the optimization is to reduce overall customer costs, while maximizing service reliability and train utilization.
  • For the asset management company, rail cars are tracked at sensor stations across the continent (Space), ETA’s are forecast-based (Time), railroad-specific actions such as switching and blocking are modeled (Activities), and the supply/demand needs of each railroad is optimized (Resources).
  • For the airline, planes and crew are assigned to minimize repositioning air miles (Space), minimize customer waiting and equipment idling (Time), respecting regulatory, union, and other constraints (Activities), and ensuring that the most appropriate plane and crew is assigned to each task (Resources).
  • For the regional agribusiness, choosing the closest processing mill (Space) and reducing the time crops are waiting to be processed (Time) is balanced with the relative efficiency of different choices crews and mills (Activities), and the availability choices in engines and rail cars (Resources).
  • For the 2020 U.S. Census, design changes include optimized routing. The new capabilities also allow quality to be infused into the process through alerts to supervisors when there is an anomaly in an enumerator’s performance (e.g., the enumerator reports having driven 100 miles in a day, but the systems show that the route could have been driven in 10 miles). In total, these design changes have the potential to save the Census Bureau an estimated $2.5 billion.

Pricing

At what price?

In most cases, prices are inputs to the models.

In some cases, however, determining the price to pay or charge is a decision that the business is making, and therefore one that optimization can assist. Some of the most sophisticated pricing changes dynamically, based on how actual consumption matches planned or forecasted consumption.

  • For the high frequency hedge fund, the optimization determines how much the fund would be willing to pay for each potential security it can acquire, and what price it would be willing to accept to unwind a position. These prices change dynamically as the market prices and forecasts for these securities change, as well as the model’s perception of the supply/demand characteristic of each security in real time.
  • For the freight railroad, optimization supports both a commissioned sales force and a self-serve web portal with dynamic pricing. “Yield management” or “revenue management” techniques are used to provide dynamic pricing designed to maximize profits rather than simply sell off capacity with fixed prices.
  • For the service network company, executive management used optimization to combine forecasted demand with the costs of multiple potential service and configuration offerings to determine the maximum revenue and profit potential.

Policy Creation

What should the rules be?

At Princeton Consultants, we have found that one of the most powerful tools for executive and operational policy creation is the combination of simulation and optimization techniques – what we call Optimizing Simulation.

Traditionally pure simulations provide the ability to create very realistic models that can provide the exact impact of various what-if scenarios, allowing the users to vary the input parameters and re-run the simulation.

By contrast, traditional optimization, presented with a set of parameters, can try billions or more combinations to show the best or highest scoring ones, given a set of rules (constraints) and a scoring technique (objective function).

In Optimizing Simulation, we get the best of both by creating a detailed simulation model of the business and then, instead of creating combinations of parameters by hand, we use optimization techniques to seek out the best combinations.

The result, we believe, is a potential revolution in strategic planning that allows businesses to evaluate their highest-level decisions faster and more rigorously.

  • For the service network company, we helped executive management to evaluate the implications of various strategic choices (the hallmark of simulation) and to provide recommendations (the hallmark of optimization) as to the best set of choices across a very large and complex set of related decisions.
  • For the high frequency hedge fund, we helped the Fund Manager analyze the implications of different risk policies under various market conditions (simulation) and recommend the best set of policies to maximize risk-adjusted return (optimization).
  • For the regional agribusiness, we helped its management rigorously analyze dozens of important strategic questions, from the “optimal” capital investments for the upcoming years, to the differences in loading, scheduling, and other operational policies.
  • For the biotech manufacturer, optimizing simulation lets executives evaluate different scenarios related to process improvement, plant scheduling strategy, and capacity planning.
  • For the e-commerce trailblazer, Princeton Consultants’ reformulated model helps executives evaluate different parameters, levels of subscriber aggregation, definitions of a “good” box, and even optimize on box value.
  • For the 2020 U.S. Census, due to the results of successful field tests driven by optimization, the Census Bureau re-engineered the business process for the enumerators.

Big Data

Extraordinarily large data sets or numbers of variables

The “big” in Big Data refers to three factors: (1) quantity of transactions, (2) speed required and (3) complexity of analysis.

We have seen how Big Data provides a golden opportunity for Big Optimization.

  • The asset management company tracks, manages and forecasts every rail car in motion at every location for every railroad in North America. Quantity: many cars, many locations. Speed: constant updates as cars are in motion. Complexity: many different car types, facilities, geographies, and cost battling with Noisy Data.
  • The airline runs one of the three largest air fleets in the world. It serves the most number of airports of these three airlines by a factor of 10. Quantity: many planes, crews, airports. Speed: need for rapid real-time responses to pilot call-ins as weather and other conditions change. Complexity: extremely high-cost vehicle assets (jets), combined with highly regulated people assets (unionized pilots), combined with varied facilities capacities and rules (airports).
  • The high frequency hedge fund uses optimization to provide low-risk trading in a quintessential Big Data environment . Quantity: thousands of securities, thousands of transactions per second. Speed: real-time, sub second solutions required. Complexity: multiple trading venues, all with different rules.
  • For the e-commerce trailblazer, choosing appropriate box configurations for millions of customers creates a problem of enormous complexity, requiring advanced modeling and algorithmic techniques to achieve reasonable solution times.
  • For the 2020 U.S. Census, on each day over the course of six weeks, tens of millions of open cases need to be efficiently assigned to enumerators so that the likelihood of a successful visit is maximized while travel time and mileage are minimized.

Noisy Data

Missing, conflicting or erroneous dataStorage or shipping containers, shelf space

By Noisy Data, we mean conditions where the input data have errors or dropouts and sometimes lag the decision.

Perhaps no complexity factor separates the academic from the real world as much as Noisy Data. Many brilliant academics, when confronted with the possibility of Noisy Data, will shrug their shoulders and say “Garbage In, Garbage Out.” Translated, this means: if you give my model Noisy Data, don’t be surprised when you get a terribly wrong answer.

In our over 30 years in business, we have yet to meet the company or the industry with perfectly complete and clean data, so we have spent a considerable amount of our time working on practical ways to reliably clean data.

  • For the asset management company, we were able to provide a sophisticated automated data cleaning facility that would use our proprietary “best current hypothesis (BCH)” techniques that piece together missing data, then potentially change the data as new data (pieces of the puzzle) arrive.
  • For the high frequency hedge fund, real-time data cleaning techniques are necessary because parties in the market motivated by competitive gaming are purposefully obfuscating their true intentions with storms of “out of the money” quotes and other techniques.
  • For the freight railroad, the large number of parties in the network contributes to Noisy Data. These parties include multiple shippers, trucking companies, facilities and even multiple railroads sharing responsibility for different legs of the same trans-continental movement. Data cleaning techniques at multiple levels are used to pinpoint potential service failures and avoid excessive “false positives.”

Stochastic

Not certain, subject to probability

By stochastic, we mean that some inputs to the problem are not single exact numbers–they are probability distributions. Many models cannot reliably handle “tail events” because they are built on a single point average of the probability distribution or similar oversimplifications.

  • As any air traveler can tell you, the availability and transit time for each plan in an airline is not a fixed number but a probability distribution, which includes factors such as equipment failures and weather delays.
  • Daily weather also plays a factor for the regional agribusiness – not merely for eventual crop yields, but also the efficiency by which vehicles move through the fields, the urgency to harvest certain areas, and even the ultimate production costs.
  • For the asset management company, demand for its assets is uncertain and must be forecasted. The standard “point” forecasts (also called “deterministic” forecasts) are much less realistic and valuable than “distributional” (or “stochastic”) forecasts, especially since the demand curve is often not a smooth, normal distribution.
  • In the case of the high frequency hedge fund, optimization needs to understand that virtually every request to buy or sell securities is a process where prices are in constant flux. Therefore, the optimal approach to balancing portfolios and achieving other risk controls will require a probabilistic strategy.
  • For the biotech innovator, the manufacturing process is highly variable, making scheduling difficult.
  • For the 2020 U.S. Census, route optimization relies in part on the probability that people in varying demographics will be home during certain day parts.

Competitive Gaming

Competitors are acting on our actions

By competitive gaming, we mean that other actors in the market are competing against our decision-making and they may change their decisions and tactics based on our choices.

  • For the high frequency hedge fund, counter parties are constantly trying to assess the relative imbalances between supply and demand, and price accordingly. Optimal buying and selling, therefore must be careful to avoid “signaling” intent.
  • The asset management company is in a “co-opetition” environment, where its major customers are sharing assets but also competing against each other. Optimizing equipment utilization for the industry means serving each client’s needs and reducing the possibility and incentives for a company to purposefully reserve equipment simply to deprive its competitor.
  • Both the freight railroad and service network company are in an industry with only a few large players and they must be careful not to compete against themselves as customers often move demand back and forth between suppliers in an attempt to ratchet down costs. Optimization helps by providing the true network cost and therefore the lowest negotiable price for each deal.

Human In The Loop

A human decision maker is in the “loop” for each decision

There are two common approaches to harnessing optimization into a decision making process in a modern organization: Black Box and Human-In-the-Loop.

By Human-in-the-Loop, we mean that the ultimate decision will be made by a person using the optimization as a tool. The complexities of this approach include:

  • How to show not just what but why an optimization model is recommending a certain action,
  • Especially when the user is non-technical (i.e., does not understand optimization)
  • How to allow the user to easily audit input data and provide appropriate ways to fix data errors for better results
  • How to allow the user to interact more with the optimization model – ideally providing ranked choices rather than a single recommendation.

Examples of Human-in-the Loop Optimization are:

  • Providing optimization-based answers to senior executives setting company strategy for the service network company
  • Shipping customers using the freight railroad’s optimization system to route and purchase freight transportation
  • Providing the airline’s planners and traffic controllers with optimized alternatives they can use to integrate with non-computerized data (such as conversations with the cockpit).

Black Box

The optimization has no human in the loop for each decision

There are two common approaches to harnessing optimization into a decision making process in a modern organization: Black Box and Human in the Loop.

By Black Box, we mean that the optimization model will be directly responsible for making a choice that was formerly performed by either a person or a prior model.

The complexities of this approach include:

  • Getting definitive rule and cost data out of subject matter experts’ heads and into the optimization model
  • Employing automated data cleaning, automated parameter tuning, and other advanced techniques that eliminate the need for human quality assurance on each transaction
  • Gaining the confidence of the sponsoring organization that there are sufficient safeguards installed; that the machine-driven approach has a lower error rate than the one it is replacing

Examples of Black Box Optimization are:

  • High-speed buying and selling for the high frequency hedge fund – providing the jump on other market participants
  • Offering web customers capacity and rates for the freight railroad without an internal salesperson – providing fast booking confirmations for customers while reducing commission costs
  • Automatically scheduling follow up lab tests for the pharmaceutical company on the same blood sample, providing faster diagnostics at low costs without requiring additional visits to the lab by the patient.
  • For the 2020 U.S. Census, in the evening preceding each day, assignments are made in over 700 different regions, which are geographically diverse, ranging from cities to rural areas. 

Static/Dynamic

The problem is solved up front but must be resolved with constantly updating information

By Static/Dynamic, we mean the optimizer presents a full solution and then resolves and makes new solutions as the data changes.

A Static/Dynamic model can provide very high degrees of solution quality, because the static part can be given many hours to come to an initial set of recommendations, whereas the dynamic resolve can ensure that real-time changes are reflected.

Static/Dynamic can provide a challenge in Human-in-the-Loop situations, as users typically do not want the optimization model to completely shuffle or change its recommendations for trivial reasons, especially when the user is spending many hours working through the optimization’s recommendations and validating them.

The complexity is that these same users do want re-solving (otherwise it would simply be a static solution, which is relatively easy) but only in cases where “it’s worth it.” In many cases, state-of-the-art technical approaches are required to fulfill this seemingly simple and straightforward request to the satisfaction of the users.

  • For the airline, a full day’s (static) schedule is created by the optimizer in the early hours of each morning, and as the day develops, the optimizer resolves and changes its recommendations as new information is known (dynamic). Because the planner/flight controller is speaking with the pilots in real time, the dynamic nature of this optimization is highly important.
  • For the high frequency hedge fund, the optimizer creates a desired portfolio (static) but then adjusts its buy/sell requests as potentially small changes occur in supply and demand (dynamic).
  • For the regional agribusiness, daily (static) plans are re-planned during the day as crops are harvested, loaded, and milled (dynamic) – squeezing out the maximum efficiency from people and equipment assets.
  • For the biotech innovator, due to the high variability of the manufacturing process and potential for product failures, the schedule is constantly updated to incorporate the latest estimates of production completion and to recover from disruptions and failures.

Real Time

The optimization must give split-second responses

By Real Time, we mean the requirement that the optimizer must re-solve as data constantly change and provide split-second responses to requests for recommendations.

The complexities of this requirement are largely technical. Designing data structures and algorithms that can give high quality answers very quickly, especially when paired with other complexity factors such as Big Data or Noisy Data, is one of the hallmarks of a state-of-the-art optimization practice.

  • As ground and air conditions change during any business day, real-time optimization allows the airline to constantly reevaluate the highest service/lowest cost assignments of pilots and planes to customers, resulting in many millions of dollars of savings.
  • Advertisers and salespeople request advertising buys for the business publication, which uses real-time optimization to reconfigure the paper and other ads on the fly to provide the best possible exposure, resulting in higher placement rates.
  • The high frequency hedge fund uses real-time optimization to track market conditions to provide the highest risk-return.
  • For the biotech manufacturer, the scheduling optimization model is used “real-time” in the simulation, so it does not limit the solution’s useability.