Skip to content
OrionStats
Module 12 · Reasoning About Effects · Business effects

Business logic — reasoning about effects, constraints and consequences to make recommendations

Turn analysis into decisions — reason about campaign, cashback, bonus, discount, partnership and commission effects; respect constraints; quantify consequences; recommend; and measure impact afterwards.

Advanced75 min
Learning objectives
  • Anticipate second-order effects of commercial levers (pull-forward, cannibalisation, gaming)
  • Incorporate business constraints and operational limits into recommendations
  • Quantify financial consequences with ranges
  • Write recommendations that are specific, prioritised and measurable
  • Design impact measurement before the decision is implemented

Every lever has side effects

Lever Intended effect Typical side effects to check
Campaigns More customers / sales now Pull-forward (sales borrowed from next weeks); discount given to customers who would have bought anyway
Cashback Repeat purchases Selection bias in evaluation; liability for unredeemed balances; margin erosion
Bonuses Motivation toward targets Gaming (pushing sales into the measured month), sandbagging plans, neglect of unmeasured work
Discounts Volume Margin loss; reference-price damage; stockpiling
Partnerships Access to new customers Cannibalisation of own channels; partner margin; data sharing
Agent commissions Partner sales effort Low-quality sales (high churn) if paid on activation only — add clawbacks or tenure-based payouts

Customer behaviour adapts: once customers learn there is a Black Friday discount, some wait for it.

Worked example: should Orion repeat a 25% Black Friday discount?

Facts (Module 8): orders/day rose from ~41 to ~57 during Black Friday 2025 at an average discount of ~22% vs ~0.5% baseline.

Reasoning chain:

  1. Incremental volume: 57 − 41 = 16 extra orders/day × 11 days ≈ 176 orders — but subtract pull-forward measured in the following 2 weeks.
  2. Discount cost on baseline demand: ~41 orders/day × 11 days × average order ~90 × ~22% ≈ 8.9k of revenue given away on orders that would have happened anyway.
  3. Incremental margin (approximation: cost ≈ 60% of full price, so margin after a 22% discount ≈ 18% of full price): 176 × 90 × 0.18 ≈ 2.9k.
  4. Budget: 80k campaign spend.
  5. Conclusion: 2.9k incremental margin − 8.9k discount on baseline orders − 80k budget ≈ −86k. On these assumptions the campaign destroyed value — unless it brought new customers with lifetime value large enough (check: how many Black Friday buyers were first-time, and how did they retain?).

The recommendation might be a smaller, targeted discount for new customers only, tested against a holdout — and a measurement plan agreed in advance.

Constraints and operational limitations

A recommendation that cannot be executed is not a recommendation. Check:

  • Capacity: warehouse throughput, call-centre agents, delivery slots.
  • Stock: will the promoted items be available? (stock-outs turn demand into frustration)
  • Budget & cash: timing of spend vs revenue.
  • Regulation & contracts: pricing rules, data protection, partner agreements.
  • Systems: can the CRM actually target the segment?

Financial consequences with ranges

Present outcomes as ranges with stated assumptions: "Expected incremental margin 15–40k (base 25k), mainly sensitive to the conversion uplift (± 2 points) and redemption rate." Show the downside: what happens if the key assumption is wrong?

Writing recommendations

Structure (one page):

  1. Answer first: "Recommend replacing the blanket 25% discount with a 15% new-customer offer and a holdout test."
  2. Why (3 evidence points with numbers).
  3. Expected impact (range) and cost.
  4. Risks and mitigations.
  5. Measurement plan: metric, control group, duration, decision rule.
  6. Next step and owner.

Data storytelling principles: lead with the conclusion; one message per chart; annotate the chart with the insight ("Pull-forward: −18% orders in the 2 weeks after"); use the audience's language (margin, cash, customers) rather than method names.

Business impact measurement

Decide how impact will be measured before implementation:

Design Strength
Randomised holdout (A/B) Strongest causal evidence
Staggered rollout (regions start at different times) Natural comparison groups (DiD)
Matched control stores Good when randomisation impossible
Before/after only Weak — confounded by trend and seasonality

Then close the loop: report realised vs expected impact, and update the assumptions used in the next decision.

Practice

Sales jumped 40% during a 1-week promotion and fell 15% below normal for the next two weeks. What is this?

Log in to answer and get feedback

Reps are paid a quarterly bonus on bookings. Bookings spike in the last week of each quarter and cancellations rise the following month. Likely explanation?

Log in to answer and get feedback

A new loyalty feature will launch to all customers next month. Which measurement design gives the strongest evidence of impact?

Log in to answer and get feedback

Write a three-sentence recommendation (answer first, evidence, measurement) for replacing a blanket 25% Black Friday discount that produced ~2.9k incremental margin on an 80k budget.

Log in to answer and get feedback

Summary

  • Anticipate side effects: pull-forward, cannibalisation, gaming, selection bias, margin erosion.
  • Check constraints before recommending; quantify consequences as ranges with assumptions.
  • Recommendations: answer first, evidence, impact, risks, measurement plan, owner.
  • Design impact measurement up front; holdouts beat before/after comparisons.