Face a realistic problem
A failed go-live, a breach on bid day, a procedure nobody follows. Each scenario starts from a kind of crisis that really happens.
Most failures aren't in the plan.
They're out of scope.
Five games · one evidence base
Construction · Engineering · Manufacturing · Safety · Technology
Out of Scope Productions
Practice the part that isn't in the plan.
Every activity is a task with a decision and a consequence, tied to a documented best practice.
The same fictional companies, crises and Database sit under everything we make. Organizations get the evidence and the debrief; players at home get the story.
Scenarios, short cases, tools and the Database, in a complete edition or discipline bundles, plus custom scenarios built on your own standards with Third Person Consulting.
AvailableSelected scenarios packaged as story-driven desktop games: the same crises, choices that carry forward, and the endings you earn.
Coming to desktopA co-operative card game for teams of up to eight plus a facilitator: ten stages, evidence cards and a scripted debrief, in discipline bundles and workshops.
Playtest 0.1Crisis & Response and its packs: quick, competitive and funny, for game nights and team socials.
Playtest 0.2A failed go-live, a breach on bid day, a procedure nobody follows. Each scenario starts from a kind of crisis that really happens.
Talk to colleagues, read what they send, and decide against the clock. The fast, confident option is usually the trap.
See the consequences, then the best practice behind each call, with its sources. Every learner leaves with a personal PDF report.
Construction, trades, facilities, manufacturing and health and safety leads. Practice the stop-work call, the pour, the evacuation and the handover, then see the OSHA rule or local code behind each one. Our largest set of scenarios.
Leaders, managers, operations, finance and project teams: the people whose calls cost money when they go wrong. Practice the go/no-go, the escalation, the estimate, the vendor and the crisis, with debrief reports and facilitated sessions for whole departments.
IT, engineering, security and PMOs. Practice the AI rollout, the breach on bid day, the go/no-go call and the cutover weekend before the real one.
Data protection officers, legal, IT and marketing teams, and anyone who buys or builds with personal data. Practice the consent call, the breach clock, the access request and the transfer, then see the GDPR practice behind each one.
IT leads, security and compliance staff, defence suppliers and anyone who signs a security questionnaire. Practice the incident bridge, the vendor login, the honest score and the build that might not be yours, then see the NIST practice behind each call.
Out of Scope Productions is owned and operated by Third Person Consulting. TPC helps organizations close the gap between what they know and what they do: learning strategy, gap assessments, best practices and training, built around our scenarios.
Scenarios, short cases and two card games, for organizations and for players at home. Each one puts people in front of a real kind of problem and asks them to decide.
Every scenario from both games is in the complete edition. Discipline bundles package the modules for one kind of team, so each customer gets what they'll use.
The Party Game is quick, competitive and funny. The Professional Edition is co-operative and evidence-based, for teams and workshops. Both are in playtesting.
Most professional failures come from the system around people: unclear authority, missing information, incentives that reward speed over care. So we don't test what people can recite. We put them under pressure and look at what they decide.
Every finding leads with what it means for you. Effect sizes and citations sit in the small print.
Body text is set in Atkinson Hyperlegible, a typeface made by the Braille Institute for readers with low vision.
Every colour-coded element also carries an icon and a label.
The Aa button makes text larger and spacing wider across the whole site, and remembers your choice on this device.
Pause any time, replay any decision, and take a series part by part. Spacing it out helps it stick.
Everything works from the keyboard, and animation switches off if your device asks for reduced motion.
The scenarios work on their own. They work better with a facilitator, a debrief and follow-up back at work. Sessions and workshops are delivered with Third Person Consulting.
TPC owns and operates OOSP. OOSP builds the scenarios; TPC fits them into the way your organization learns, with learning strategy, best practices and training.
Buying for an organization? Company details, privacy, accessibility, security and delivery are all on one page for your vendor and IT reviewers. For procurement ↗
Tell us about your team and the problem you're trying to solve: a facilitated scenario session, a workshop or a custom scenario.
hello@oosp.fun
Learning strategy, knowledge and process gap assessments, best practices and training for organizations of every size. Owns and maintains the Best Practices and Standards Database.
That's us: we build the scenarios and games, drawn from TPC's Database.
Out of Scope Productions is a Third Person Consulting company. TPC owns and operates OOSP: we are the studio that makes Project: Countdown!, Near Miss! and the other games. TPC maintains the Best Practices and Standards Database behind them. OOSP is how Third Person Consulting delivers scenario-based learning, backed by the world's largest gamified best practices database Tier D · Unverified
We searched and couldn't find a bigger one. We can't prove there isn't one, so we graded our own claim the way the Database grades everyone else's.
The name is the idea. Projects rarely fail on the work everyone planned for. They fail on what was out of scope: the vendor who goes quiet, the approval nobody owned, the risk that was somebody else's problem until it wasn't. Most training skips that part. We build activities that let people practice it.
We hold ourselves to three rules. Make it engaging enough that people choose to finish it. Build it on evidence, and be honest about how strong that evidence is. Design it so every learner can use it.
OOSP is our shorthand. The full name is Out of Scope Productions.
It started as a question on an index card: what if project and crisis management were a role-playing card game? The cards became playable scenarios, every call got traced to a documented practice, and the Database grew into games, card games, tools, briefs and workshops.
I made Project: Countdown! and the Database because of a pattern I kept seeing in six years of project strategy and implementation work in IT and AECO. Solo practices, small firms and global companies had the same problems, and the "best practices" meant to fix them were often ones nobody had checked.
Twenty-two years of research, projects and knowledge management taught me to go to the sources. Every entry in the Database says where it comes from, how strong the evidence is, and what we're not sure about.
I spent five years designing and running role-play scenarios. As a neurodivergent creative, I know people learn in different ways; digital scenarios let each person practise the decision in a way that works for them.
“I don't accept a best practice just because everyone else does.”
Read the full profile at Third Person Consulting ↗The consulting practice that owns and operates OOSP: learning strategy, knowledge and process gaps, and best practices across departments, delivered with our scenarios.
Entry-level briefs for anyone starting a change, and no homework. The Ten Stages overview walks through the Database's lifecycle from Diagnose to Adapt, with a brief for each stage; topic briefs follow one idea across departments, newest first. Every claim ties to an entry in the Database.
Why a change is a cycle, not an event: what each stage is for, what it hands the next, where changes usually break, and how to size the effort. All ten stage briefs open from here.
For the engineer who is also the project manager and the office: a few procedures, a contractor system, hold points where work gets covered up, and a learning plan you'll keep.
Learning · Change · Projects · Software · AIThe famous statistics that didn't survive a trip to the source, what each one actually rests on, what to say instead, and five questions to check a number yourself.
Technology · AI · Knowledge · LegalA pre-launch checklist: governed sources, a gold-set test, citations and "not found", permissions and leakage, client contracts, and logs handled as people data.
Technology · Plants · Utilities · ProjectsReadiness you test rather than survey, a contingency for the system that runs wrong, five ticket classes, and a hypercare that ends on data instead of a date.
Safety · Projects · Cyber · Contracts · PlantsRisk is mostly decided before anything goes wrong: base rates, controls where mistakes can't be undone, a way back, signals worth watching, and how to decide when it happens anyway.
All topic briefsEach guide is written to be read in one sitting and used for months. The small codes after a paragraph, like CM-RSK-02, are Database entries: open one to see the sources, how strong the evidence is, and what we're not sure about. Where a guide says "practise it", the scenario named puts the same decision in front of you under pressure.
The guides are general guidance for professional teams, not legal, engineering or safety advice. Where a rule depends on your jurisdiction or contract, the guide says so.
A new system, a new procedure, a new site or a new role isn't adopted on launch day. It lands, or doesn't, across ten stages that start before anyone buys anything and keep going long after the training ends. This read explains the whole cycle: what each stage is for, what it hands the next, where changes usually break, and how to size the effort. Each stage then has its own brief.
Turn a request into a problem you can test, keep three explanations open, and never baseline at the worst month.
The future workflow, who decides, what information moves, and success conditions set before anything is bought.
Managers ready to coach, partners aligned, governance live, knowledge captured, and the fallback rehearsed.
Worked examples, practice from memory, spacing and feedback: the methods that work, and the myths to retire.
First use on real work: bounded, supported at the moment of need, with deviation treated as data.
Turn claims of competence into evidence, and check that the checks themselves can fail.
Roll out in waves, measure depth rather than logins, and close the old route.
Help inside the work, people who help, and reading every question as data.
Measure at the right level, against a comparison, and report money as a range with the method shown.
Turn lessons into changes, decide whether to continue or stop, and start the cycle again.
Most organizations plan a change as an event: a launch date, a training week, a go-live weekend. The event is real, but it's a small part of what decides whether the change works. What decides it is mostly invisible on the project plan: whether anyone checked that the problem was the one being solved, whether success was written down before the money was spent, whether managers were ready to coach, whether people practised before they had to perform, and whether anyone was still paying attention six months later. #52 · #63
The Database organizes its advice into ten stages because that's where the evidence clusters. Each stage answers one question, and each produces something the next stage needs. Skip one and the work doesn't disappear; it turns up later, at a worse moment and a higher price. #52 · #9
The stages are also a cycle. Adapt, the last one, uses everything learned along the way to change the system, and the next change starts with a better diagnosis than the last. An organization that runs the cycle well doesn't just complete one change; it gets better at change. #64 · KIN-CAP-03
The ten stages fall into three movements. The first decides what the change is. The second builds the capability to do it. The third makes it normal and keeps it alive.
Measure is set up in Define, when the baseline and success conditions are written down, and runs alongside Support. Adapt feeds the next Diagnose. #10 · #14 · DGN-BSL-01
Decide (Diagnose, Define, Prepare). Most failed fixes were aimed at the wrong problem, and the most common causes of project failure people cite are scope creep, overload and unclear goals, not technology or budget. These three stages exist to stop that: test the problem before naming the fix, write down the future work and the success conditions before buying anything, and get people, partners, systems and knowledge ready before go-live. PM-1 · DGN-CNS-01 · PM-13 · #52
Build (Learn, Apply, Verify). The learning research is unusually clear about what works for adults learning technical work: worked examples, practice from memory, spacing, feedback and whole real tasks. But learning that never meets real work fades quickly, so Apply puts it to use within days, in bounded conditions. Verify then turns claims of competence into evidence; a certificate, a signature or a green status report is a claim, not proof. #2 · LSC-TRF-03 · SOP-TRN-01 · #28
Sustain (Adopt, Support, Measure, Adapt). A change is adopted when the old way has quietly disappeared, and that rarely happens on its own. Support catches everything training couldn't anticipate. Measure separates the change's effect from everything else going on. Adapt turns all of it into decisions, including the decision to stop. This is the movement most often left unfunded, because budgets tend to end at launch. #63 · CM-DIG-03 · #22 · #64
The stages connect through handoffs. When a stage is skipped, the next one starts without what it needs, and usually doesn't notice until much later.
| Stage | Hands the next stage | Skip it, and… |
|---|---|---|
| Diagnose | A tested problem, its causes, and what's still unknown | The fix is aimed at the symptom, often "more training" |
| Define | The future workflow, decision rights, information flows and success conditions | Every stakeholder carries a different version of "done" |
| Prepare | Ready managers, partners, systems, knowledge and a rehearsed fallback | Training competes with the backlog the launch created |
| Learn | People who can do the work on an unseen task, without help | Completions are reported as capability |
| Apply | First real use, with deviations recorded as data | Skills fade before they're used; people revert under pressure |
| Verify | Evidence that people, systems and the checks themselves work | The organization relies on a claim |
| Adopt | The new way as the only way, with depth of use measured | Half the teams keep the old spreadsheets |
| Support | Answers at the moment of need, and a record of the questions | Workarounds grow and the workflow drifts |
| Measure | What changed, against a baseline, with the method shown | Nobody can say whether it worked |
| Adapt | Changes made, and a better diagnosis for next time | The same lessons are logged again on the next project |
DGN-CNS-01 · PM-19 · TAR-PLN-02 · LSC-MSR-01 · LSC-TRF-03 · SOP-GOV-05 · GOL-ADP-01 · #21 · DGN-BSL-01 · SOP-GOV-06
The same failure points turn up across software rollouts, plant changes, new procedures and new sites. None of them is dramatic. Each is a stage done on paper instead of in the work.
Not every change needs every stage at full depth. A new report template and a new payroll system both pass through all ten stages, but the template can move through several of them in an afternoon. The question at each stage is the same; the effort scales with the consequence of getting it wrong and how hard it is to undo. GOL-HYP-08 · QMS-ARC-03
Two stages are never optional, whatever the size: Diagnose, because a fix aimed at the wrong problem wastes everything after it, and Measure, because without it nobody can tell whether the change worked or should be stopped. PM-1 · #10
Each stage brief is written to be read in one sitting: what the stage is for, the practices that matter most, figures you can reuse, a "by team" section for technology, projects and construction, plants, safety and finance, and a short list of things to keep current. Read the brief for the stage you're in, then the one after it, because most problems show up at the handoff. #52
If you're starting something new, read Diagnose and Define before anything is bought. If a change has stalled, start with Adopt and Measure, then work backwards: the stage where the evidence runs out is usually where the cycle broke. The topic briefs then follow single ideas, such as onboarding, go-lives and risk, across all ten stages.
The cycle in one line. Find the real problem, say what done looks like, get ready, build skill that lasts, use it on real work, prove it, make it normal, support it, measure it honestly, and change the system based on what you learned.
Each topic brief follows one idea across departments and all ten stages. Newest first.
For the engineer who is also the project manager and the office: a few procedures, a contractor system, hold points where work gets covered up, and a learning plan you'll keep.
Learning · Change · Projects · Software · AIThe famous statistics that didn't survive a trip to the source, what each one actually rests on, what to say instead, and five questions to check a number yourself.
Technology · AI · Knowledge · LegalA pre-launch checklist: governed sources, a gold-set test, citations and "not found", permissions and leakage, client contracts, and logs handled as people data.
Technology · Plants · Utilities · ProjectsReadiness you test rather than survey, a contingency for the system that runs wrong, five ticket classes, and a hypercare that ends on data instead of a date.
Safety · Projects · Cyber · Contracts · PlantsRisk is mostly decided before anything goes wrong: base rates, controls where mistakes can't be undone, a way back, signals worth watching, and how to decide when it happens anyway.
Guide and tool · Onboarding · Rollouts · LeadershipLearn, contribute, own: how to stage the first 90 days in a new role, site or system, with six sector templates and a builder that turns them into your own plan.
Construction · Contracts · BIM · Quality · FinanceBase rates, delivery models, information requirements, production control, payment law and decisions under pressure, in the order a project meets them.
Learning · Workforce · AI · OperationsTreat learning as a system, not a catalogue. What the evidence supports, what to retire, how to build capability across disciplines, and an 18-month roadmap.
People · Safety · Projects · TechnologyFrom the week before day one to the moment someone can be trusted with real work: role analysis, two tracks for new and experienced hires, practice that sticks, and how to know it worked.
Most failed fixes were aimed at the wrong problem. This brief shows how to turn a request into a problem you can test, keep more than one explanation alive, and hand the next stage a diagnosis it can trust.
Every change starts with someone noticing that something isn't working. A plant manager sees rework climbing. A project director sees requests for information piling up on one subconsultant. An IT lead hears that nobody uses the new document system. A safety manager watches near-miss reports dry up and isn't sure whether that's good news. The temptation at this moment is to name a fix straight away: more training, a new tool, a tighter procedure, a reorganization. Diagnose is the stage where you resist that temptation long enough to find out what is actually wrong. PM-1 · #13
The stage has a clear job. By the end of it you should be able to state the business or safety outcome at stake, the size of the gap, who it affects, what is causing it, which causes training can fix and which it can't, and how ready the organization is to change. You should also be able to say what you still don't know. That last part matters more than it sounds: a diagnosis that admits its gaps is far more useful than one that sounds certain and turns out to be wrong. DGN-CNS-01 · #13
Diagnosis is not only for big transformations. The same habits apply to a single crew that keeps missing hold points, a finance team whose month-end close always runs late, or a software team whose releases keep breaking. The scale changes; the discipline doesn't. In a small firm, a diagnosis might be a two-hour conversation and a page of notes. In a capital programme it might be a formal front-end review with an independent reviewer. What both share is that they separate the problem from the solution and test the first before buying the second. PM-9 · DGN-FED-01
A useful definition of done: the diagnosis is finished when someone who wasn't involved could read it, check its evidence, and reach the same conclusion, or see exactly where they would disagree. Everything in this brief serves that standard.
The best evidence on how diagnosis fails comes from medicine, where errors are studied carefully, and from public investigations of accidents and failed projects. Both point the same way. Failures are rarely caused by a lack of method. They're caused by settling on one plausible story too early, trusting indicators that measure the wrong thing, and skipping the one test that would have shown the system wasn't ready. DGN-FRM-02 · DGN-BIAS-01
A study of 100 diagnostic errors in internal medicine found that only seven were "no-fault" errors alone. In the other 93, the researchers found an average of nearly six contributing factors per case. System factors, such as poor handovers, missing information or workload, contributed to about two-thirds of the cases. Thinking errors contributed to about three-quarters. Many cases had both. The single most common thinking error was premature closure: settling on the first plausible answer and no longer considering alternatives. DGN-FRM-02 · DGN-BIAS-01 · PM-2
Medical diagnosis, not organizations; applying it at work is inference. The lesson that transfers: expect several causes, and expect at least one to sit in the system rather than the person. DGN-FRM-02
Organizational failures show the same pattern at a larger scale. Canada's Phoenix pay system was framed around savings, budget and timeline; its only pilot was cancelled, the system was never tested as a whole, and more than 100 functions were deferred to fit the budget without anyone re-examining the expected benefits. A year after launch, about half of employees had errors in a single pay run. Before the 2005 Texas City refinery explosion, a very low personal-injury rate was read as a sign of good process safety, while earlier surveys and audits had already flagged deep problems. In both cases, the information existed; the diagnosis didn't use it. DGN-RDY-03 · DGN-RDY-04 · DGN-BSL-02 · DGN-DATA-02
Four traps account for most of the damage:
Start by rewriting the request as an observed gap against a named outcome. State the outcome that matters, the level it's at now, the level it should be at, who is affected and since when. Then strike out any word that names a fix. If the statement only works with the fix still in it, you have a solution looking for a problem. DGN-FRM-01 · IDE-DES-01
"Our site engineers need a two-day course on the inspection and test plan."
"On our three Ontario sites, 14% of concrete pours since March went ahead without the hold point being released, against a target of zero. It's concentrated on two sites and started when the new ITP template replaced the old one."
The second version does three things the first can't. It names a measurable outcome. It tells you where to look, because two sites and a template change are clues. And it leaves the cause open: the template, the inspector's availability, the schedule pressure or the engineers' knowledge could all be responsible, and a course only fixes one of them. DGN-FRM-01 · QMS-ITP-02
When the request comes from a sponsor, diagnose the request itself before accepting it as the brief. Write down the decision the sponsor faces and what evidence would show the requested solution addresses it. Agree at the start that the diagnosis might find the request wrong. This is easier to say in the first meeting than after you've spent three weeks confirming the wrong thing. Where a solution is legally mandated, such as certified training for working at heights, diagnose the barriers to it working rather than whether to do it. DGN-FRM-04 · SOP-CAN-02
On capital projects, the equivalent of framing is front-end definition. Industry research on large industrial projects finds that how complete the front end was at authorization is one of the strongest predictors of whether a project delivers what it promised. Owners can score this with tools such as the Project Definition Rating Index: in NASA's building projects, those scoring 200 or less averaged about 1% over budget and 2% late, against 6% over and 11% late above that line. The association is correlational, but the element-level scores also show where owner, designer and contractor rate the same item differently, which is a diagnosis in itself. OGV-FEL-01 · DGN-FED-01 · DGN-FED-02
Once the problem is framed, list the candidate causes and sort them into families. Six families cover most workplace problems, and more than one usually applies. Record evidence for and against each. Recommend training only where a knowledge or skill gap is actually shown, and only alongside fixes for the other families that turned out to matter. DGN-FRM-02 · IDE-DES-02
In the medical study behind Figure 1, most thinking errors came from gathering and combining information, not from missing knowledge. Treat "they need training" as one hypothesis among six. DGN-FRM-02 · #15
A practical way to use the families: for each one, ask the people closest to the work for a recent example. If engineers can explain the hold-point rule perfectly but released pours anyway because the inspector was booked two days out, the gap is capacity and process, not knowledge. If they can't explain it, knowledge is in play, but probably not alone. DGN-GAP-01 · DGN-DATA-04
Two cause families deserve extra suspicion. The first is the behaviour explanation. "Unsafe acts cause most injuries" is a line that has circulated for decades; before Texas City, the refinery's safety campaign was built on it, and investigators found systemic process-safety and management failures instead. Treat any behaviour explanation as a hypothesis that needs its own evidence. The second is capacity, because it is the family people are least likely to volunteer. If a business case assumes each person will handle twice the workload after a new system arrives, measure what they handle now, in the current system, before relying on the assumption. DGN-FRM-03 · DGN-TNA-03
For judgment-heavy tasks, where experts can't easily explain what they do, simple interviews will miss most of it. Structured methods such as cognitive task analysis, which combine interviews with observation and real cases, produce much better training content for troubleshooting, estimating and emergency command, though they take time and need genuine experts. Don't build content from expert self-report alone: experts routinely leave out steps they no longer notice they take. DGN-TNA-01 · DGN-TNA-02 · #18
The single most useful habit in diagnosis is to keep at least three competing explanations open until the evidence tells them apart. For each, write what would confirm it and, more importantly, what would rule it out. Close an explanation only after you've checked the evidence against it. This is the direct remedy for premature closure. It also changes how you collect data, because you go looking for the observation that would separate the explanations instead of the one that confirms your favourite. DGN-BIAS-01 · DGN-CNS-03
| Explanation | Would confirm it | Would rule it out | Status |
|---|---|---|---|
| Engineers don't know the new hold-point rule | They can't describe it; errors spread evenly across sites | They can explain it; errors cluster on two sites | Ruled out |
| The new template hides the hold point | Errors began with the template; the field is on page 4 | Sites on the old template show the same rate | Supported |
| Inspector availability forces a choice | Pours booked with under 48 hours' notice to the inspector | Inspector was on site for most released pours | Open |
Keep the "open" row visible in the report. "Not yet established" is a finding, not a failure. DGN-CNS-01
Three more checks protect the diagnosis from its own blind spots:
Be careful with the tools that promise to find "the" root cause. Five Whys is fine for quick triage of a simple, single-cause fault, but it oversimplifies complex events: the Lac-Mégantic investigation produced 18 causal and contributing findings. For serious events, use methods that trace how control failed through management and oversight, with a small interdisciplinary team. Debiasing checklists sound helpful, but the best-known ones were never rigorously tested, and a small trial found they didn't reduce errors. Pair them with structural changes and feedback rather than relying on them alone. QMS-RCA-03 · DGN-RCA-01 · DGN-BIAS-03 · QMS-RCA-06
Finally, log every warning. Keep a simple register of every warning or dissenting signal: who raised it, who received it and what was decided. Phoenix's executives didn't grasp the importance of warnings that the pay centre and the system weren't ready, and the decision to go live wasn't documented. A register costs minutes and makes that failure much harder to repeat. DGN-BIAS-04 · PM-22
A diagnosis needs numbers, and numbers chosen at the wrong moment can mislead badly. The most common mistake is baselining at the moment the problem was noticed. Problems get noticed at their worst. A metric picked because it looked bad will usually drift back toward its normal level on its own, a statistical effect called regression to the mean, and any fix introduced at that moment will appear to have worked. DGN-BSL-01 · #14
Use several measurements before the change, ideally a full operating cycle, and a comparison group where you can. DGN-BSL-01 · DGN-BSL-04 · LSC-MSR-05
Three further rules keep baselines honest:
Before commissioning any new diagnosis, check whether earlier ones were acted on. If an audit two years ago found the same problem and nothing changed, the problem is no longer information. It's decision-making, and the diagnosis should say so. DGN-DATA-02 · DQ-LRN-04
The last part of diagnosis asks whether the organization is ready to change. Readiness has two parts that are worth reporting separately: commitment, whether people value the change, and confidence, whether they believe they collectively have the knowledge, resources and time to make it work. Report how readiness varies between sites, teams and roles, not just the average: a wide spread is itself a warning. DGN-RDY-01 · DGN-RDY-02 · #38
Surveys and status reports aren't enough on their own. Where the stakes are high, confirm readiness with a behavioural test: a pilot, a dress rehearsal or a parallel run, made a gate criterion. If the test is cancelled, require a documented impact assessment and a fresh decision by the accountable executive. Whenever scope is cut later, re-run the benefits and readiness diagnosis; a cut that seems administrative can remove exactly the function the benefits depended on. DGN-RDY-03 · DGN-RDY-04 · #1
Then write it up in a form the next stage can use. A good diagnostic report separates three things: what was found, with evidence from at least two kinds of source; what is opinion, labelled as such; and what is not established, with what would settle it. It reports the hypotheses that failed as well as those that survived, says which cause families training can address and which it can't, and summarizes the warnings register. Keep the diagnostician independent of whoever will sell or deliver the solution, and plan who hears the findings, when, and through which channel, because the diagnosis itself changes the system. DGN-CNS-01 · DGN-DATA-03 · DGN-CNS-02 · DGN-CNS-04
A note for small organizations. None of this requires a consultant or a software platform. A thirty-person firm can do a sound diagnosis with a one-page gap statement, a short table of three explanations, a dozen recent examples from the people doing the work, and a baseline drawn from records it already keeps. The discipline is the point, not the paperwork. LSC-ORG-03 · PM-1
The checklist. Write the gap without the fix. Diagnose the request. Sort causes into six families and expect several. Keep three explanations open and write what would rule each out. Test the test; check that closed means done. Get a second look when it matters. Log every warning. Never baseline at the worst moment. Report found, opinion and not established. Then hand on to Define.
A diagnosis tells you what's wrong. Define turns it into a picture of the work as it should be: the workflow, who decides, what information moves, and how everyone will know it worked, written down before anyone buys anything.
Diagnose ends with a clear account of the problem and its causes. Define answers the next question: what will the work look like when the problem is solved? That means the future workflow, the roles in it, who has the authority to make which decisions, what information each step needs and where it lives, and the conditions under which everyone will agree the change has succeeded. #63 · #62
Define is where most projects quietly lose. Research on project success keeps finding that the causes people cite most often are scope creep, overload and unclear goals, not technology or budget, and that relational factors such as shared understanding between sponsor and team outrank technical ones. Those are all failures of definition. When nobody wrote down what success looked like, every stakeholder carries their own version, and the project spends its life negotiating between them. PM-13 · PM-8 · PM-19
The stage has a simple test. By the end of it, could a new team member, a partner firm or an auditor read one short document and understand what is changing, who decides what, and how the result will be judged? If not, the definition isn't finished. In a small organization that document might be two pages. On a capital project it may be a set of contract schedules and an execution plan. The questions are the same. PM-12 · #14
If any box is empty or says "TBD", the definition isn't done. Box 1 should match the gap statement from Diagnose. DGN-FRM-01 · PM-12
The most common definition error is to describe the technology instead of the work. "Implement the new common data environment" is a purchase, not a definition. "Every model and drawing is shared, reviewed and published through one platform, with the reviewer's approval recorded before anything reaches site" is a workflow. The difference matters because adoption is a socio-technical change: roles, incentives, procedures, information and skills all move together, and a definition that only names the tool leaves all of them unplanned. #63 · ITG-CHG-01
Write the future workflow as a sequence of real tasks. For each step, say who does it, what they need, what they produce and who receives it. Pay special attention to the hand-offs between people, teams and tools, because that's where most information is lost. If the change spans several applications, define the workflow across them rather than one product at a time; people work in workflows, not in products. #20 · CM-DIG-01
Where AI is part of the change, define the human-AI hand-off before anyone is trained on the tool. Which steps does the AI draft, which does a person check, and which remain entirely human? Who owns each AI output? Before any of that, decide whether AI should do the task at all, and map where it is reliable and where it isn't, because capability is uneven even within one job. #35 · AIG-01 · PRM-BIZ-05 · PRM-EVD-07
Two cautions help keep definitions honest:
Match the size of the definition to the shape of the change. A small procedural change needs a short definition; a change that alters how several departments work together needs a fuller one, and possibly a named framework for handling complexity. Hybrid approaches, mixing planned and iterative work, are a legitimate end state rather than a compromise, as long as the definition says which parts are fixed and which will be learned as you go. PM-11 · PM-20 · PM-19
The second most important box on the canvas is decision rights. Write down who recommends, who decides and who must be consulted for every significant decision the change will create: approving a model for construction, releasing a hold point, stopping work, changing a procedure, accepting a system at handover, pausing an AI tool. Investigations of serious failures keep finding the same gap. At Grenfell Tower, the architect, contractor and cladding subcontractor each assumed someone else was responsible for fire safety. Nobody had written it down. DQ-UP-07 · ITG-GOV-01
| Decision | Project director | Information manager | Design lead | Site manager | Client |
|---|---|---|---|---|---|
| Approve the BIM execution plan | Decides | Recommends | Consulted | Consulted | Consulted |
| Move a model from "shared" to "published" | Consulted | Decides | |||
| Accept information at handover | Recommends | Consulted | Decides | ||
| Stop work on a safety concern | Anyone on site, then resolved in writing | ||||
No controlled evidence favours one framework (RACI, DACI and others) over another; what matters is that each decision has exactly one decider, and that stopping work is never reserved to a single person. DQ-UP-07 · SOP-USE-03 · DQ-RT-02
Then line the contracts up with the decision rights. A responsibility chart that contradicts the contract will lose every argument. Make sure the contract says who can stop work, change the design and approve temporary works, and remember that delivery models change who can act: oversight gaps tend to follow the contract. Under each standard form, know who administers the contract and who is responsible for what. DQ-UP-10 · VCM-FRM-02 · CM-DEL-01
Two refinements make decision rights work under pressure. First, sort decisions by whether they can be undone. Reversible decisions can be delegated and made quickly; irreversible ones, such as pouring concrete, signing a lump sum, publishing a model or deleting data, deserve options, independent review and kill criteria. Second, pre-assign crisis decisions before the crisis: who can shut down a system during a cyber incident, who orders an evacuation, who pauses an AI tool today. Decisions that have to be negotiated in the moment are usually made late. DQ-UP-01 · CYBER-01 · AIG-14 · HX-03
On projects with several firms, decision rights need agreement across organizational boundaries. Align partners on digital goals, roles and success conditions at the start, and bring public agencies in early where they have a role, agreeing who commands when. #62 · SEM-13 · BM-REQ-01
Every step of the future workflow consumes information and produces it. Define what each step needs, where it comes from, who owns it and what form it takes. In construction and asset management, this discipline has a formal shape: requirements cascade from the organization (what the business needs to know about its assets) to each project and then to each exchange between parties. Defining them in that order keeps a model full of what the owner will actually use, rather than whatever the authoring tool produced by default. BM-REQ-01 · FAM-SYS-01 · FAM-DAT-01
Specify information by need, not by a generic detail level. "Level of development 400" says little about what the facility manager needs for a pump; a level-of-information-need table that lists the properties, documents and geometry each object requires says a great deal. Set the project information standard and production methods before tender, so bidders price what you'll actually ask for. BM-REQ-07 · BM-REQ-06 · BM-USE-01
Outside construction, the same principles apply in different words:
Finally, define how the documentation fits together. A clear hierarchy, with policies, procedures, work instructions and records each in their own lane, prevents the common failure where a project plan quietly overrides an enterprise procedure or a checklist contradicts the method statement it was meant to support. SOP-SYS-02 · SOP-SYS-05 · SOP-SYS-03
Define success and capture a baseline before launch. Projects that set success criteria and a measurement baseline before work starts are more likely to succeed, and those that don't can only ever report activity: courses delivered, licences issued, logins recorded. Write success conditions as measurable outcomes with an owner, not as intentions. "Improve coordination" isn't a condition; "design-attributable RFIs on the next two projects fall by a third from this year's baseline, owned by the design manager" is. #14 · PM-12 · OGV-BIZ-02
Set conditions at more than one level, because each level answers a different question and each moves on a different timescale. Plan the evaluation now, including where the data will come from, rather than bolting it on after launch. IDE-DES-07 · #10 · IDE-EV-01
Satisfaction scores belong nowhere on this ladder: how much people liked a course barely relates to what they learned. LSC-MSR-01 · LSC-MSR-03 · IDE-AEC-02
Be honest about uncertainty in the numbers. Forecast cost and schedule from a reference class, meaning what similar projects actually cost and took, and then adjust for what's different about yours. Show the business case at the confidence it really has, as a range, and uplift early estimates for the optimism that early estimates reliably contain. A single confident number at the definition stage is usually the best case in disguise. CM-RSK-02 · PM-17 · OGV-BIZ-03 · OGV-BIZ-01
Expect a dip. Many changes make people slower before they make them faster, and industrial AI tends to follow a J-curve in which costs come first and gains later. Write the expected dip into the success conditions so that week six doesn't look like failure. #11 · DM-AIM-01
A definition needs a way to say no. Define each gate's deliverables and criteria in advance, before anyone sees the pitch, and make "recycle" and "stop" real options alongside "go". Gates that can only open aren't gates. Separate the people who decide from the people who advocate, and commission independent assurance at the points where mistakes become expensive. OGV-GATE-02 · OGV-GATE-01 · OGV-GATE-03
Agree stop rules when the project is approved: the cost, date or benefit conditions under which it comes back for a decision. Stop rules agreed in advance are far easier to honour than ones invented when the money is half spent, because sunk costs and personal commitment drive people to keep going. Generic warnings about escalation don't help; specific decision rules do. OGV-ESC-02 · PM-5 · OGV-ESC-01
Then control scope. Freeze it at a defined point and govern every late addition against criteria, and put work into a constrained window, such as a plant shutdown, only if it genuinely needs that window. Standardize the highest-consequence processes first rather than everything at once; a short list of procedures that are actually followed beats a long list that isn't. TAR-SCP-01 · TAR-SCP-02 · SOP-SYS-06 · SOP-DES-07
Keep governance proportionate. A thirty-person firm doesn't need a portfolio board; it needs one person who owns the decision and a written note of what was agreed. A large owner may need formal gates and independent review. Over-governing a small change is as wasteful as under-governing a large one. ITG-GOV-05 · OGV-ASR-02 · QMS-ARC-03
Take a 300-person mechanical and electrical contractor whose diagnosis found that design-attributable RFIs had doubled in two years. The causes were mixed: a new subconsultant working to a different modelling standard, a coordination meeting that no decision-maker attended, and junior coordinators who had never been shown how to group clashes. Training was part of the answer, but only one part. DGN-FRM-02 · BM-COO-04
The definition that followed filled the canvas like this. Outcome: design-attributable RFIs back to the level of two years ago, about eleven a month, within three projects. Future workflow: each discipline checks its model against the agreed rules before sharing; federation runs weekly; clashes are reported as clusters with an owner and a date; decisions are made in the meeting, not after it. Decision rights: the design lead decides when a model is published; the information manager can refuse a delivery that fails the checks; the project director decides disputes between disciplines. Information: one modelling standard written into every appointment, including the subconsultant's, with a level-of-information-need table for the objects that matter most to the installers. BM-QA-02 · BM-COO-02 · BM-COO-03 · BM-REQ-07
Success conditions were set at three levels: coordinators running the clash process unaided by day 60; nine in ten deliveries passing the automated checks first time by month four; RFIs at target by the third project. Gates and stop rules: if deliveries were still failing checks at month three, the subconsultant's appointment would be reviewed rather than the training repeated. Nothing in the definition said "buy software" or "run a course", though both happened later. Because the work was defined first, each purchase had a job to do and a way to tell whether it had done it. BM-COO-05 · OGV-ESC-02 · #10
Common mistakes to look for when you review a definition like this:
Define hands Prepare a short, specific document: the canvas in Figure 1, filled in. It should be readable in ten minutes and checkable by someone who wasn't in the room. Before you hand it on, test it against four questions:
Explain the change in the terms of the people who will live with it. A definition written for the steering committee rarely persuades a site crew or a finance team. Anchor it to the mandates and standards people already recognize, and say what's in it for them in their own words; relevance in the learner's terms is one of the cheapest ways to win early commitment. #26 · #27 · KIN-GOV-01
The checklist. Fill all six boxes. Describe the work, not the tool. Define the human-AI hand-off. Give every decision one decider, and let anyone stop work. Line the contract up. Cascade information requirements from the organization down. Set success conditions at five levels, with a baseline and an owner. Forecast from a reference class, as a range. Define gates that can say no, and stop rules before you start. Then hand on to Prepare.
Prepare is everything that has to be true before people can do the new work: managers ready to coach, partners aligned, systems secured, knowledge captured, and a plan for the day things go wrong. Most of it is invisible when it's done well.
By the time a change reaches Prepare, the problem is diagnosed and the future work is defined. Prepare gets the organization ready to do that work. It covers people, managers, partners, technology, governance, knowledge and project conditions: a long list, which is why this is the stage with the most advice in the Database and the one most often squeezed when a date is fixed. #52 · #9
The cost of skipping preparation shows up later and larger. Training delivered after a system goes live competes with the backlog the system created. Security settled after a pilot has to unpick data already shared. A partner who learns the information requirements at the first model delivery reworks everything. A plant shutdown that starts with incomplete work packs spends its first days waiting for permits. In each case the work had to be done anyway; it was just done at the most expensive moment. #52 · #31 · TAR-PLN-02
A useful finish line for Prepare: on the first day of the change, nobody has to stop and ask for something that could have been arranged in advance. People have access, managers know their role, partners know the rules, the fallback is rehearsed and the knowledge that matters is written down. The rest of this brief takes those in turn.
Scale the weeks to the change. A procedure update might compress this into two weeks; a plant system into twelve months. The order matters more than the length. TAR-PLN-01 · DGN-RDY-03
Put training ahead of the technology rollout, not after it, and time each module close to when people will first use it. Training delivered months early is forgotten by go-live; training delivered after go-live is delivered to people who are already behind. The best window is usually the few weeks before first use, with short refreshers spread over the weeks after. #52 · LSC-TRF-03 · #3
Prepare the people who will do the work before you hire around them. Upskilling existing staff is usually faster and cheaper than recruiting, and they already carry the organization's context. Build role-based paths rather than one programme for everyone: a site engineer, a design coordinator and a finance analyst need different things from the same rollout. Diagnose what each person already knows, and run separate tracks for newcomers and experienced people, because heavy guidance that helps novices actively gets in the way of experts. #5 · #25 · LSC-CLT-02
The single most neglected preparation is for managers. Supervisor support and the climate of the team are among the strongest predictors of whether training is used on the job. Give managers a short, specific role: what their people have practised, what to give them to do next, what to watch for and how to coach the new way of working. Then give them time for it. A manager told to support adoption while carrying a full load will, reasonably, prioritize the load. #53 · LSC-TRF-04 · PRM-LRN-04
Choose champions carefully, and expect modest gains. Across 35 health-care studies, champions went with better adoption in some organizations, but effects on practice and outcomes were mixed or unmeasured; neither volunteering nor appointing several is a proven formula. Pick people colleagues already trust, write down what they do, teach them to explain the reasoning behind a workflow rather than the clicks, give them protected, non-billable time, and judge them on errors and rework, not sessions held. Leadership commitment needs to be visible too: people watch what senior staff actually use, not what they announce. #8 · LSC-MNT-05 · #7
Two practical preparations are easy to forget:
Few changes in construction, engineering or manufacturing stay inside one organization. Subconsultants, subcontractors, software vendors and clients all have to be ready too, and their readiness varies. Assess partner digital maturity before relying on it, and write what you need from them into their appointments: the information standard, the delivery formats, the review cycles and the security rules. Requirements that live only in a kick-off presentation are optional in practice. #6 · BM-REQ-04 · NIST-SUP-02
Then test the methods during mobilization, before production. On a BIM project, that means a trial exchange of models and data between every party, checked against the agreed rules, in the first weeks. On a manufacturing change, it means a first-article run with the supplier's real parts. On a software rollout, it means a test integration with each connected system. Each test finds the problems while they are cheap. BM-REQ-05 · #56 · KIN-CAP-04
A few contract preparations pay for themselves many times over:
Clear security and data governance before piloting, not after. Once a pilot has shared data with a new platform, unwinding it is slow and sometimes impossible. Decide which data may go into which tool, keep a record of what any AI is trained on and how long data is kept, and settle privacy duties before launch rather than after a complaint. #31 · DG-01 · PRM-SEC-01 · AIG-11
| Area | Ready means | Typical gap |
|---|---|---|
| People | Role-based paths delivered close to first use | One course for everyone, months early |
| Managers | A written coaching role and time to do it | "Support the rollout" on top of a full load |
| Partners | Requirements in appointments; methods tested at mobilization | Requirements shown in a kick-off deck |
| Technology | Access, security and integrations tested in a sandbox | Permissions sorted out on day one |
| Governance | Data rules, decision rights and change control live | Policies drafted, not yet operating |
| Knowledge | Experts captured; one current version published | "Ask Dave" as the documentation |
Use it as a readiness board: each row gets an owner and a red, amber or green rating, backed by evidence rather than opinion. SEM-30 · TAR-PLN-01
Prepare the technology for learning as well as for production. Use a safe sandbox before AI or any new system touches production work, so people can make mistakes where mistakes are free. Error-management training, in which learners are encouraged to make and learn from errors in a safe setting, improves transfer to new situations. #44 · LSC-TRF-05 · #12
The security basics are unglamorous and decisive. Phishing-resistant multi-factor authentication on all remote and privileged access. No standing administrator rights. Networks segmented, with plant and building controls separated from office IT. Default credentials replaced before handover. Every new AI agent given its own scoped identity, with "don't touch production" enforced as a permission rather than written in a prompt. CYBER-04 · NIST-PRT-01 · CYBER-08 · NIST-OTX-02 · AGT-01 · AGT-02
Governance has to be operating, not just written. A policy binder is not a control. Build governance as repeatable processes with named owners, check that controls actually run rather than that policies exist, and put a version and a date on every framework you cite. ITG-GOV-02 · AIG-16 · NIST-GOV-04
Every change depends on knowledge that sits in a few people's heads: how the old system handled the odd cases, why a procedure has that extra step, which supplier always ships late. Capture tacit knowledge before experts leave, and before the change makes their knowledge harder to reach. Facility management alone expects a large share of its workforce to retire within a decade, and the same pattern runs through trades, plants and engineering offices. #18 · FAM-WRK-01
Capture in the flow of work rather than in a separate documentation project. Ask the experienced technician to record how they actually do the isolation while they're doing it. Have the senior estimator talk through a real bid. Pair veterans with newer staff, so judgment moves one way and digital fluency the other. Mentoring is a retention lever as well as a learning one. KIN-CAP-02 · FAM-WRK-02 · LSC-MNT-02
Don't rebuild what already exists. Convert useful internal material into training and job aids rather than buying generic content, because people learn faster from their own organization's templates and standards. Buy generic tool basics where they exist, and keep firm-specific workflow and standards training in-house. #17 · #16 · LSC-SFT-03
Then make the knowledge findable. Publish one current version at the point of use, with a version stamp on anything printed or offline. Point any enterprise AI only at governed sources and make it cite them. Check knowledge readiness at project mobilization: does the team know where the lessons from the last similar project live, and has anyone read them? SOP-DIG-01 · SOP-DIG-03 · KIN-TWN-01 · KIN-CAP-04
The part of preparation people most dislike is planning for the change going wrong. It is also the part that most often decides whether a bad day becomes a bad month. Start with a pre-mortem: before committing to the plan, imagine it has failed badly and ask everyone to write down why. The often-quoted claim that pre-mortems make forecasts "30% more accurate" overstates the original study, which found people generated about 30% more reasons for failure. That is still the point: more reasons, surfaced while they can still be acted on. CS-03 · PM-22
Then build and rehearse the way back:
Exercise the plans with the people who will run them. Tabletop exercises build shared understanding and confidence; technical drills test whether things actually work. Add unscripted events: a satisfying exercise is not the same as readiness, and real incidents rarely follow the script. Pre-brief people before stressful simulations, and give them a way out. CYBER-13 · GL-16 · CS-05 · CS-09
The six preparation areas apply everywhere, but each kind of team meets them differently. A few examples show where the effort usually goes. #63 · #13
For a new platform, preparation is mostly invisible plumbing: identity, access, integrations, data rules and support. Give every account a named owner and the minimum rights it needs, connect the new system to the identity provider before anyone logs in, and agree how learning and usage data will be handled before collecting any. If the platform uses AI features, keep a register of what each one does and who checks it. Plan the help-desk load for the first month, which is usually far higher than anyone budgets. NIST-PRT-01 · IDE-DAT-01 · IDE-DAT-04 · #51
On a project, preparation is mobilization. Build learning and the new practices into the project plan from the first week, confirm the information requirements and execution plan with every party, test the exchange methods before production, and set up the notice register, the constraint log and the lookahead before the first deadline arrives. Set up short-interval planning with a dedicated facilitator, because pull planning that depends on volunteers rarely survives the first schedule crisis. #56 · BM-REQ-05 · VCM-ADM-01 · CM-LEAN-01 · CM-LEAN-04
In a plant, preparation means operators who were involved in the design, procedures validated against the way the work is actually done, and the start-up planned from the start of the project rather than in its final weeks. Co-design new systems with operators and make them explain themselves; a screen operators don't trust becomes a screen they work around. Resource commissioning and start-up as their own team, early, and track long-lead materials on the critical path. DM-WRK-02 · PRM-MFG-03 · TAR-SU-01 · TAR-SU-03 · TAR-PLN-03
For safety, preparation is mostly about the plans for bad days: exercised with every agency that would respond, with communications that don't depend on public mobile networks, and with the decision to evacuate or shut down assigned in advance. Keep emergency plans current and exercised rather than filed, and design out hazards at the design stage where you can, because a hazard removed on drawings never needs a procedure on site. SEM-12 · SEM-10 · HX-03 · CM-SAF-03
In finance, preparation means knowing who owns each number, training everyone who will operate a control before they operate it, and keeping a backup for every critical role. When a new system arrives, the controls built into the old one need to be rebuilt and re-tested, not assumed. FIN-ORG-02 · FIN-TAL-02 · FIN-TAL-07 · FIN-TEC-04
Prepare ends with a readiness gate. Hold it against measurable criteria agreed in advance, and confirm readiness with a behavioural test wherever the stakes justify it: a pilot, a dress rehearsal, a parallel run or a commissioning test of every system before the public or the plant depends on it. Status reports say what people believe; tests show what is true. SEM-30 · DGN-RDY-03 · SEM-11
Start small where you can. Piloting on one live project before scaling, and starting with one bounded workflow with human review, consistently beats a big-bang launch. Pilots surface the integration problems, training gaps and workload effects that no plan predicts, while the cost of finding them is still low. #1 · #12 · OC-14
Read schedule pressure as a warning sign rather than a reason to skip the gate. Pressure to go live on a fixed date is when preparation gets cut, and on construction sites schedule pressure is itself a known safety signal. If the gate fails, the honest options are to fix and re-test, to narrow the scope, or to delay. Going live anyway is a decision, and it should be recorded as one, with its owner. CM-SAF-06 · DGN-BIAS-04 · OGV-GATE-01
For small organizations, the same discipline fits on one page: a list of the six areas, a name against each, a red, amber or green rating with one line of evidence, and a short rehearsal of the riskiest step before the real thing. That page, honestly filled in, is worth more than a thick plan nobody tests. ITG-GOV-05 · #1
The checklist. Train ahead of the rollout, close to first use. Give managers a coaching role and time. Pick volunteer champions with protected hours. Put requirements in partner contracts and test methods at mobilization. Clear security and data governance before the pilot. Open a sandbox. Capture what the experts know. Run a pre-mortem; rehearse the fallback and the restore. Pass a behavioural readiness test. Then go to Learn.
Learning research is unusually clear about what works for adults learning technical work: worked examples, practice from memory, spacing, feedback and whole real tasks. This brief turns that evidence into choices you can make on Monday.
Learn is where people build the knowledge, skill, judgment and ability to work alongside new technology that the defined future work requires. It sits after Prepare because the conditions for learning matter as much as the content: managers ready to coach, a sandbox to practise in, real templates to practise with. It sits before Apply because learning that never meets real work fades quickly. #52 · #2
This stage is not "deliver the training". A course is one possible tool. Many of the most effective forms of learning happen at work: a supervised first attempt, a worked example from last month's project, feedback on a real model, a debrief after a difficult pour. The test of this stage is simple: can people do the new work to the required standard, on a task they haven't seen before, without help? Satisfaction ratings, completions and attendance don't answer that question. LSC-MSR-03 · LSC-MSR-01 · IDE-DES-06
The good news is that the research on adult learning for technical work is unusually consistent, and the methods that work aren't expensive. The bad news is that the methods that work best often feel worse to learners, look worse in training, and score lower on satisfaction surveys. Most of this brief is about choosing what works over what feels good. LSC-PRC-05 · #19
One way to read learning research is to ask where the typical person trained one way lands among people trained another way. If two methods are equally good, the typical learner lands in the middle: better than half of the comparison group. Strong methods move that typical learner well above the middle. LSC-EVD-02
The line marks "no difference". These are education results: the whole-task figure comes mostly from higher-education programmes, and nobody has yet shown it's better than alternatives inside engineering firms. Expect smaller effects for complex field procedures, and treat any learning effect as a learning effect, not a productivity forecast. LSC-DES-02 · LSC-CLT-01 · LSC-MOD-01 · LSC-MYTH-01 · LSC-EVD-02
A handful of principles carry most of the weight:
Expect modest results from good design and smaller ones from complex procedures. A recent re-analysis of multimedia research found real effects that are smaller than first reported and shrinking over time. That's not a reason to give up on design; it's a reason not to promise "89% better retention" to anyone. IDE-MM-04 · LSC-EVD-02
For anyone learning a task for the first time, use the sequence with the strongest evidence in technical training: a complete worked example, then partly completed tasks, then independent ones, with feedback at every step. For a BIM coordinator that might mean watching a senior colleague set up a federation and run a clash test, then finishing a half-built test, then running one alone on a live model. For a maintenance technician it might mean studying a completed isolation, then completing one with the last steps missing, then doing one from scratch under supervision. LSC-CLT-01 · LSC-TRF-06 · #59
Fade fastest for experienced people on well-structured procedures; for them, heavy guidance gets in the way. LSC-CLT-05 · LSC-CLT-02 · #60
Two refinements make the sequence work at work. First, start from what people already know. A short diagnostic task on day one tells you who can skip the early steps. Experienced people need problems and references rather than walkthroughs, and a direct conversation about where their old habits will clash with your way of working, such as drafting habits carried from 2D CAD into model-based work. LSC-CLT-02 · LSC-CLT-03
Second, use your own material. People learn faster from their organization's standards, templates and past projects than from generic examples, and the transfer to the job is shorter. For fixed procedures, the closer the training is to the real task, the better. Keep firm-specific workflow and standards training in-house; buy generic tool content only where it really is generic. #16 · LSC-TRF-02 · LSC-SFT-03
Teach the reasoning and deliver the steps when they're needed. The reasoning behind a code clause or an analysis method builds judgment and deserves teaching time. The click-by-click steps in a software tool change with every release and belong in a job aid inside the tool, not in anyone's memory. Teach complexity; look up specifics. LSC-DES-04 · #22
The single most reliable finding in learning research is that retrieval strengthens memory more than review. For technical work, retrieval doesn't mean a multiple-choice quiz. It means doing the task from memory: setting up the model, isolating the pump, completing the permit, pricing the change, then getting feedback on what was missed and trying again. Corrective feedback followed by another attempt is where much of the learning happens. LSC-PRC-01 · #19 · #60
Evenly spaced refreshers work about as well as "optimal" algorithms, so don't pay extra for them. Time each refresher to when the skill will actually be used if you can. LSC-PRC-02 · LSC-PRC-03 · GL-10
Mix problem types when the skill is choosing which method applies. Troubleshooting a pump, diagnosing a model error or picking a contract clause all require recognizing which kind of problem you're facing. Practising one type at a time feels efficient but teaches people to apply a method without first deciding whether it fits. Interleaved practice teaches the decision. LSC-PRC-04 · LSC-TRF-07
Expect the best methods to feel worse. Practising from memory, spacing sessions and mixing problem types all lower performance during training while improving retention and transfer. Learners left to themselves will choose rereading and massed practice, because those feel fluent. Build retrieval and spacing into the programme rather than leaving them to choice, and warn learners and managers in advance that the dip in practice scores is a good sign. LSC-PRC-05 · LSC-PRC-06
Let people make errors safely. Training in which learners are encouraged to make mistakes and learn from them, in a sandbox model, a test controller or a mock tender, improves transfer to new situations. Debrief after the work, not just after the course: a short, structured debrief after a real task turns experience into learning. LSC-TRF-05 · #44 · SOP-TRN-02
Years on the job are not the same as deliberate practice. An experienced person can repeat the same habits for decades without feedback that would improve them. Engineer practice deliberately: target the specific weaknesses, give fast feedback and raise the difficulty. This matters most for senior staff learning new tools, who often keep inefficient strategies because their results still look fine. LSC-EXP-01 · LSC-EXP-02 · LSC-SFT-01
Much of what matters in construction, engineering and operations is judgment: deciding whether a crack is benign, whether a schedule is believable, whether to stop work. Judgment is learned by making decisions with feedback. Authentic decision simulations with structured debriefs are one of the few ways to give people that practice before the real stakes arrive. Make the decisive information salient rather than making the scene realistic, pair every game or simulation with a debrief, and add short reflection pauses during play. #61 · GL-01 · GL-05 · GL-08
Rehearse decisions under time pressure, not only with time to spare, and raise the pressure in steps. Practise short, structured call-outs for situations where information has to move quickly between people. Trust gut feel only where the cues are valid and learnable: expert intuition is reliable in familiar situations with regular feedback, and unreliable in new ones. CS-01 · CS-02 · CS-07 · CS-04
AI has changed what learning has to cover. Build universal AI fluency before specialist tracks, teach data literacy before advanced AI use, and teach human-AI collaboration rather than prompt entry alone. Most important, make AI failure detection a practised skill: train people to find planted errors in AI output, recompute numbers outside the model and check every citation. #58 · #41 · #43 · #45 · PRM-LRN-02
Guard AI tutors carefully. In a field experiment, students who practised with an unrestricted AI assistant did better while using it and worse than others once it was taken away; a guarded tutor that gave hints and checked steps, rather than answers, avoided most of the harm. Step-level feedback is what makes tutoring work. Check proficiency periodically with the AI switched off, and protect the early-career practice that used to come from routine work AI now absorbs. LSC-MOD-03 · LSC-MOD-04 · PRM-LRN-01 · #54
Choose the format by the task, not the trend. Use VR where real practice is dangerous or impossible: confined spaces, fall protection, crane familiarization, a building that doesn't exist yet. Across studies with real-world outcomes, VR worked about as well as conventional training, with a small edge for physical tasks. It's poor value for knowledge content such as standards, and in construction safety the evidence for fewer incidents is still thin. LSC-MOD-01 · IDE-MM-09 · IDE-AEC-01
Treat microlearning as a format, not a method. Short modules suit spaced refreshers and job aids; they don't suit complex judgment, and the "eight-second attention span" used to sell them is folklore. Gamification has small, conditional effects: prefer challenge, narrative and choice to points and leaderboards, and let the outcome be the score. LSC-MOD-02 · LSC-MOD-05 · GL-12
Get the multimedia basics right, because they're cheap: cut decoration and background music, put labels on the diagram rather than in a legend, break complex material into learner-paced pieces, never paste the narration onto the slide, and caption everything. Use on-screen presenters only when their gestures add information. IDE-MM-01 · IDE-MM-02 · IDE-MM-03 · IDE-MM-06 · IDE-MM-07 · IDE-MM-08
Retire the myths. "Learning styles" don't improve learning when matched. "Rewires the brain" is true of all learning, including bad learning, and proves nothing. "Only 10% of training transfers", "the 10,000-hour rule" and generational learning labels have no sound basis. Andragogy's maxims (explain why, use real problems, respect experience) are good practice, but not science, and newcomers still need structure. LSC-MYTH-01 · LSC-MYTH-02 · LSC-TRF-01 · LSC-EXP-03 · LSC-AGE-01 · LSC-EVD-01
Design for the pace of the learner, not stereotypes about age. Older trainees tend to take longer in training, yet age is essentially unrelated to job performance. Offer controllable pace, replayable demonstrations and extra practice to everyone who needs them, without stigma, but don't promise they close the gap: a 2022 meta-analysis found pacing and teaching approach didn't change age differences as expected. Diagnose prior knowledge and confidence instead of assuming, and don't mistake speed for competence. LSC-AGE-02 · LSC-AGE-04
The principles are the same across teams; what changes is what "a whole, real task" looks like and where practice can safely happen. LSC-DES-02 · LSC-DES-03
For software and digital tools, teach workflows across tools, not isolated products, and teach the strategy behind them, not just the commands. People who learn the buttons without the strategy keep slow methods for years, because their results still look acceptable. Train with your own standards and templates, put the click-steps into job aids inside the tool, and give people a sandbox where errors cost nothing. For AI tools, teach people to brief the tool, give it sources and check its output, and practise finding planted errors. #20 · LSC-SFT-01 · #16 · #44 · PRM-LRN-02
For engineers and designers, much of what matters is judgment and verification. Build verification habits into analysis and AI training: hand checks, mesh-sensitivity checks and load-path sanity checks, and the same scepticism toward any AI output. Graduates learn mostly from coworkers, so prepare the people they sit beside, and match development to career stage: early-career engineers need feedback and graded responsibility, principals need enough of the new tools to judge the work they sign. LSC-SFT-04 · LSC-MNT-03 · LSC-AGE-03 · DQ-AI-03
On site, the best learning happens close to the work: a worked example from this project, a supervised first task, a debrief after the pour. Retire read-and-sign as proof of competence, and use error-based scenarios for the situations crews rarely meet. Refresh skills by how fast they decay, and remember that training sticks when the workplace backs it, which means foremen and superintendents who expect the new way. SOP-TRN-01 · SOP-TRN-03 · SOP-TRN-04 · SOP-TRN-05
For technicians, combine operational technology, data and troubleshooting in the same programme, using vendor-neutral credentials and apprenticeships where they exist. Fault-finding is a judgment skill, so vary and mix the practice: the same fault every time teaches recognition of one fault, not the skill of telling faults apart. Use VR or simulators for the hazardous tasks nobody can practise on a live unit. DM-WRK-04 · DM-WRK-05 · LSC-PRC-04 · LSC-MOD-01
For roles that act under pressure, practise under pressure, raising it in steps; practise the call-outs; and run exercises with unscripted injects followed by structured debriefs. Pre-brief people before stressful simulations and give them a way out, because frightened learners remember the fear, not the lesson. Practise in scenarios rather than annual click-through modules, and reward reporting rather than only avoiding mistakes. CS-01 · CS-07 · CS-09 · NIST-LRN-01 · NIST-LRN-02
Learn hands Apply a group of people who have practised the new work, from memory, on tasks like the real ones, with feedback, spread over weeks. Before you hand on, check three things. Has each person shown they can do a representative task without help? Do their supervisors know what they practised and what to give them next? And is there a refresher plan that carries into the first weeks of real work? LSC-MSR-03 · LSC-TRF-04 · LSC-PRC-02
Pair the training with a live project quickly. Skills that aren't used within days or weeks of learning start to fade, and the first real use is where most of the remaining learning happens. Interdisciplinary cohorts help here: when engineers, coordinators, site staff and finance learn a new workflow together, they also learn each other's hand-offs. #2 · #47 · LSC-TRF-03
For small organizations, none of this needs a learning platform. A worked example from your own files, a half-finished version for the learner to complete, a check against your own standard and three short refreshers in the calendar will outperform most purchased courses. #17 · #16 · LSC-CLT-01
The checklist. Build on whole, real tasks. Diagnose prior knowledge and run two tracks. Show, then fade. Practise from memory with feedback and another attempt. Space refreshers at about one, three and eight weeks. Mix problem types for judgment. Expect the best methods to feel harder. Let people err safely. Simulate decisions and debrief them. Guard AI tutors and test with AI off. Choose formats by task, and retire the myths. Then go to Apply.
Apply is the first time new capability meets real work. Done well, it happens quickly, in bounded conditions, with help at the moment of need and a clear way to say "this doesn't fit". Done badly, it's where good training goes to die.
Apply moves new capability into real work, preferably quickly and in bounded conditions. It's the bridge between practice and normal operations: the first coordination meeting run the new way, the first pour inspected against the new plan, the first month-end closed in the new system, the first maintenance job done with the new procedure. #2 · #56
Two forces work against this stage. The first is time. Skills that aren't used within days or weeks of learning start to fade, so a gap of two months between training and first use quietly undoes much of the investment. The second is pressure. The first real use usually happens on a live project with a deadline, where the new way feels slower and the old way is right there. Without deliberate support, people revert, and the organization concludes that the training didn't work. LSC-TRF-03 · #11
The goal of Apply is not perfection on day one. It's a controlled first use that produces evidence: what worked, what didn't, where people needed help and where the defined workflow itself was wrong. That evidence feeds Verify, which confirms people can do the work to standard, and Adapt, which fixes the system. #64 · SOP-USE-01
Pilot on one live project before scaling, and when AI is involved, pilot one bounded workflow with human review. A bounded first use limits the cost of the problems you'll certainly find, keeps support close, and gives you a clean comparison with work that hasn't changed yet. In manufacturing, studies of digital initiatives find that scaling, not the technology, is usually the bottleneck; a pilot that never had a path to scale is a demonstration, not a step. #1 · #12 · DM-ARC-07
Move out a ring only when the current one has produced evidence, not when the calendar says so. Irreversible steps stay supervised longest. #1 · #44 · DQ-UP-01
Within the bounded first use, apply three rules that come from field procedures and work in any setting:
Make the first use part of the project, not an extra. Build the new practice into project mobilization, so it's in the plan, the budget and the meeting cadence from the start. A new coordination process that depends on someone remembering to run it will not survive the first deadline. #56 · BM-COO-04
Almost every meaningful change makes people slower before it makes them faster. Plan for an initial productivity dip, tell people and managers it's coming, and protect it in the schedule. Industrial AI follows a similar J-curve, with costs and disruption first and gains later. A rollout that treats the dip as failure tends to abandon the change at its lowest point, just before it would have paid off. #11 · DM-AIM-01
An illustration, not data. The size and length of the dip vary; set success conditions that expect it, so week six isn't judged as failure. #11 · #14
Three things shorten and soften the dip:
Make inefficiency visible as well as the dip. Experienced people can keep slow methods for years because their results look fine. Showing them the time their own tasks took with the old and new methods is more persuasive than any course outline. LSC-SFT-02 · LSC-SFT-01
During Apply, most questions arise in the middle of a task. Put performance support in the flow of work: short, searchable guidance inside the tools people already use, at the step where they need it. A two-minute answer inside the CDE, the CMMS or the ERP is worth more than a course the person would have to leave the work to find. Provide one discovery point for everything else, so nobody has to remember which of four portals holds the answer. #22 · IDE-TEC-05 · #30
Use checklists for the steps that matter, designed the way that works. Read-and-do checklists suit novices and rare tasks; do-and-confirm checklists suit experts. Keep only the items that kill or cost, because a long checklist trains people to tick without looking. And remember that a checklist works through how it's introduced, with the people who use it, not because it exists. SOP-CHK-01 · SOP-CHK-03 · SOP-CHK-02
Pre-task planning deserves its own mention on construction sites and in plants. A short pre-task briefing works best when it trains people to spot hazards in the task in front of them, not when it's a form signed at the start of the shift. Keep one current version of every procedure at the point of use, with version stamps on printed and offline copies, so nobody follows a superseded method. SOP-CHK-05 · SOP-DIG-01 · SOP-DIG-03
For complex procedures, small practical aids help: marking your place in a long procedure, putting the warning before the step rather than after it, and designing for the field with pictures and plain words. These are cheap, and they prevent a surprising share of errors. SOP-USE-02 · SOP-DES-04 · SOP-DES-05
The first real use of any new procedure will find cases it didn't anticipate. What people do at that moment decides whether the procedure improves or quietly dies. Treat deviation as data: a record of where the written method and the real work disagree. A normal, routine shortcut is a warning, not a convenience, because it means the procedure and the work have drifted apart. SOP-USE-01 · SOP-USE-05 · DGN-GAP-01
Stop-work is one layer of protection, never the only one, and every stop is resolved in writing, never by a phone call. SOP-USE-03 · DQ-RT-01 · DQ-RT-02 · SOP-GOV-03
Supervisors play a particular role here. Supervisors verify in the field, not from the office: walking the work is how they see where the procedure and the site disagree. A ticked box isn't a verified outcome, and a field change is a design change, with the same review. Where several employers share a site, coordination between them needs its own procedure, because each firm's method statement assumes the others aren't there. SOP-USE-04 · SOP-CHK-04 · SOP-GOV-03 · SOP-USE-07
When the stakes are high and time is short, the crisis rules apply. Put one accountable lead in charge, brief facts, interpretations and projections separately, act on partial information only where the step can be undone, and keep safety decisions apart from commercial ones. In a plant incident involving control systems, safety comes before the investigation. DQ-RT-06 · DQ-RT-04 · DQ-RT-14 · DQ-RT-10 · NIST-OTX-03
AI tools are often the first new capability people apply, because they're easy to start using. That ease is the risk. A few rules make first use safe and useful:
Measure whether the AI actually helps. Speed-ups that people report and speed-ups that are measured often differ, and in one controlled study experienced developers working on familiar code were slower with AI assistance. Expect modest, context-dependent gains, and check them in your own work. PRM-EVD-08 · SWD-AI-02 · SWD-AI-01 · FAM-WRK-03
Bounded first use, help at the moment of need and deviation as data apply everywhere. The boundary and the risks differ by team. #1 · SOP-USE-01
For software, the bounded first use is a staged release: a small batch, to a small group, with automated health checks, a kill switch and rollback ready. Mitigate first and fix the code later when something breaks, run incidents with defined roles, and give communication its own owner so the people fixing the problem aren't also answering the questions. A first use without a way back isn't bounded, however small the audience. SWD-REL-01 · SWD-REL-03 · SWD-INC-01 · SWD-INC-03 · SWD-INC-04
On a project, first use usually means the first package, zone or pour done the new way. Run the coordination meeting with decision-makers present, report clashes as clusters with an owner and a date, and enforce the information states so that "shared" and "published" mean something. Commit only to work that's ready, and record why commitments were missed; those reasons are the deviations of the planning process and the best guide to what to fix. BM-COO-04 · BM-COO-02 · BM-CDE-01 · CM-LEAN-01 · CM-LEAN-03
In a plant, the first use of a new procedure or system often coincides with a start-up or a shutdown, the most hazardous hours of operation. Treat every temporary change through management of change, keep the first runs supervised, and use short control cycles so problems surface in hours, not weeks. If AI is involved, use it for knowledge and documents, never for control or safety functions. TAR-SAF-01 · TAR-SAF-04 · TAR-EXE-01 · DM-AIM-04 · PRM-MFG-01
For safety-critical work, first use is where stop-work authority gets tested. Set the stop-work conditions in advance, make sure new people know they'll be backed for using them, and resolve every stop in writing before work restarts. Pre-task planning should train people to spot the hazards in the task in front of them, rather than to sign a form. SOP-USE-03 · DQ-RT-02 · SOP-CHK-05 · CS-06
In finance and contracts, first use is often a deadline: the first prompt-payment clock under new legislation, the first close in a new system, the first notice under a new form. Run statutory clocks exactly, keep the notice register live, and administer contracts after signature, not just before. Confirm any change to payment details by phone, on a number you already had, because the first weeks of a new system are when fraudsters find gaps. VCM-PAY-01 · VCM-ADM-01 · VCM-ADM-04 · NIST-PRT-02 · FIN-CTL-03
A 200-person precast manufacturer introduced a new inspection-and-test plan for its yard. Instead of switching all three production lines at once, it started on one line for one product family. The quality lead checked the first ten elements personally, the foreman kept a one-page aid at the casting bed, and every deviation went into a shared log. In the first fortnight the log showed the same problem four times: the plan asked for a measurement the yard's tools couldn't take at that stage. The plan was fixed in a day. Had all three lines switched at once, the same problem would have appeared on every line, and the plan would probably have been blamed rather than fixed. QMS-ITP-01 · SOP-USE-06 · SOP-USE-01 · #1
Apply hands Verify a group of people who have done the new work for real, in bounded conditions, with records of what happened: first-work checks, deviations, debrief notes and the questions they asked. Before handing on, make sure that evidence exists and is in one place. Verify needs it to judge whether people can do the work to standard, and Adapt needs it to fix what the first use exposed. SOP-USE-01 · #64
Look especially for patterns. If three people asked the same question in the first week, that's a gap in the performance support or the training, not three individual weaknesses. If the same deviation appeared on two sites, the procedure is probably wrong. These patterns are the most valuable output of Apply, and they're easy to lose if nobody collects them. #32 · KIN-SIG-01 · DGN-RCA-04
For small organizations, Apply can be as simple as choosing the next real job, assigning an experienced person to check the first attempt, putting the job aid on the shared drive, and spending ten minutes afterwards on what to change. The bounded first use and the debrief matter far more than any platform. #1 · SOP-TRN-02
The checklist. Apply within days of learning. Pilot on one project, in widening rings. Verify the first work, then keep watching. Keep irreversible steps supervised. Expect the dip and protect it. Put help in the tools, at the step. Treat deviation as data; let anyone stop work, and resolve it in writing. Brief AI properly, give it the source and verify its output. Collect the patterns. Then go to Verify.
Verify confirms that people, teams, systems and the checks themselves can do what the change needs, to standard, before the organization relies on them. A certificate, a signature or a green status report is a claim. This stage turns claims into evidence.
Verify answers one question: can the people and teams do the required work to the required standard, and how do we know? It comes after Apply because the best evidence of competence is real work, done in bounded conditions. It comes before Adopt because the organization shouldn't make a new way of working normal until it's sure the way works and the people can do it. #28 · #49
Verification is easy to confuse with paperwork. A signed acknowledgment that someone read a procedure, a training completion record, a certificate from a course provider, a status report marked green: each of these is a claim that capability exists. None of them shows it. Verification means observing the work, or its results, against a standard agreed in advance, and being honest about how strong the evidence is. SOP-TRN-01 · SOP-TRN-06 · SOP-GOV-05
The stage covers four things that need verifying, in increasing order of how often they're forgotten: individuals, teams, systems and processes, and the checks themselves. That last one sounds abstract until you read accident investigations, where a check that passed for the wrong reason appears again and again. DGN-GAP-02 · QMS-AUD-02
Verify competency before project assignment. Before someone works alone on anything consequential, watch them do a representative task to the organization's standard and confirm they can interpret the result. For model-based work, that includes whether they understand what the model is telling them and who is accountable for it, not only whether they can operate the software. The principle is "show me, don't tell me". #28 · SOP-TRN-06 · BM-ROL-04
A judgment scale, not measured data. Each step up adds something the one below lacks: a real task, no help, a standard, and time. LSC-MSR-01 · LSC-MSR-03 · LSC-MSR-04 · LSC-CPD-01
A few rules keep people-verification honest:
Watch for overconfidence. People who have just finished a satisfying training session or game often overestimate what they learned. Debrief from objective records rather than memory, and check performance separately from confidence. GL-13 · GL-02 · GL-14
Individual competence doesn't guarantee team performance. Hand-offs, communication and shared understanding fail in ways no individual test will find. Rehearse as a team: tabletop exercises for decisions, technical drills for systems, dress rehearsals for cutovers. Add unscripted events, because a smooth, satisfying exercise is not the same as readiness, and practise the short, structured call-outs that move information under pressure. GL-16 · CS-05 · CS-07 · CYBER-13
Test and commission every system before people depend on it. In events, that means every safety and communication system before the public arrives. In plants and buildings, it means handing over and accepting systems one at a time, with operations accepting each. In digital services, it means checking that service-level objectives still reflect what users care about. SEM-11 · TAR-SU-02 · SWD-SLO-03
Rehearse the manual fallback for critical tasks that a new system or AI now supports, and test full restores rather than backup jobs. The time to discover that nobody remembers the paper process is during a rehearsal, not during an outage. #46 · CYBER-10 · HO-13
Verify procedures before release, too. Test the procedure with the people who will use it, in conditions like the real ones, before it's issued. A procedure that's followed exactly and still fails needs redesign, not retraining. SOP-DES-06 · SOP-GOV-11 · SOP-DES-01
The most dangerous verification failures are checks that pass when they shouldn't. Test the test: for every check the organization relies on, walk through how it's physically performed and ask whether anything other than the target could make it pass. At Lac-Mégantic, the hand-brake test was done with the locomotive's air brakes still applied, so it looked as if the hand brakes alone would hold the train. DGN-GAP-02 · DGN-AST-01
Independent review is only as good as its reach. Scope independent design review to the hazard: the reviewer must be qualified for the work, and the scope and budget must cover the critical elements and the construction stages. At the FIU pedestrian bridge, the peer reviewer wasn't qualified by the state for that category of work and lacked the budget to check the connection nodes or the construction stages, and calculation errors went through. An unqualified reviewer isn't a second set of eyes. DQ-UP-05 · SOP-GOV-02
| Level | Who checks | Example | Blind spot |
|---|---|---|---|
| Self-check | The person who did the work | A modeller runs the agreed rules before sharing | Their own assumptions |
| Independent check | A qualified person outside the task | A peer reviewer checks the connection design | Whatever is outside their scope or budget |
| Independent assurance | Someone reporting to the decision-makers, not the project | An owner's reviewer reports unedited to the board | Whatever they aren't told |
Each level needs the one below it. For high-consequence work, the scope of each level should be written down and checked against the hazards. BM-QA-02 · DQ-UP-05 · OGV-ASR-01 · OGV-GATE-03
Paperwork can be checked and still be wrong. A ticked box isn't a verified outcome. A clean audit isn't proof that there are no nonconformities. And a management system can exist on paper while the work happens some other way, a pattern quality researchers call decoupling. Audit the work, not the binder: trace one feature from requirement to as-built and see whether each step really happened. SOP-CHK-04 · QMS-AUD-02 · QMS-RWK-05 · SOP-GOV-05 · SOP-GOV-09
Verify materials and data that can't be checked later. For safety-critical materials, don't rely on the manufacturer's own test or certificate alone; the Grenfell Inquiry found manipulated tests and certification bodies that were vulnerable to manipulation. Check that a certificate is accredited and covers your use, not just that it exists. DQ-UP-12 · AIG-02 · AIG-03
Finally, verify that corrective actions worked. Don't close an action on the evidence that it was implemented; define the effectiveness check before closing it, and have management review ask whether closed actions actually fixed anything. QMS-CAP-03 · QMS-CAP-02 · QMS-ARC-05
Information products need verification as much as people do, and they're easier to check automatically. On BIM projects, write the data requirements in a machine-checkable form from the start, use rule-based checkers for what those can't cover, and check against the owner's asset information requirements at every stage gate, not only at handover. Expect the common data environment to fall short of the standard and design checks for the gap. BM-QA-01 · BM-QA-03 · BM-QA-04 · BM-CDE-04
Check a schedule's health before you trust its dates. Run the standard checks for missing logic, hard constraints and high float, and read high float as missing logic rather than spare time. A critical path built on gaps looks fine until the gaps close. PCF-SCH-02 · PCF-SCH-03 · PCF-SCH-01
Simulations and digital twins need a budget for verification, validation and stated uncertainty. A model that has never been compared with the real asset is a picture, not evidence. DM-TWN-05 · FAM-TWN-06
AI output needs its own verification discipline:
Some verification is non-negotiable. In process plants, the pre-start-up safety review must be complete before hazardous materials are introduced, and a unit should never be started with known faults in its safety instruments. At Texas City in 2005, a tower was started up with a level indicator, a level alarm and a control valve known to be malfunctioning; it overfilled, and the explosion killed 15 people. Start-ups and shutdowns are among the most hazardous hours in any plant. TAR-SAF-02 · TAR-SAF-03 · TAR-SAF-01
The same logic applies wherever a mistake can't be undone: a lift, a pour, an energization, a go-live that replaces a system with no way back. Define the verification required before the irreversible step, make it a hold point, and track every time the hold point is bypassed. A bypass is information about the system, not just about the person. QMS-ITP-02 · QMS-ITP-04 · DQ-UP-01
Give gates teeth. A verification gate that has never stopped anything is decoration. Write its criteria in advance, separate the people who decide from the people who advocate, and make "not yet" a respectable answer. When the gate does say no, record the decision and its reasons; when someone overrides it, record that too, with their name. OGV-GATE-01 · OGV-GATE-02 · OGV-GATE-03 · DGN-BIAS-04
Get a second look where the stakes justify it. In one study, about 1 in 5 referral diagnoses ended distinctly different after an independent review. Engineering, finance and safety decisions are not medicine, but the lesson transfers: an independent look, at the evidence rather than the conclusion, changes more than people expect. DGN-BIAS-02 · OGV-ASR-01
Every team verifies people, systems and checks, but the evidence that counts differs. #28 · SOP-GOV-05
For software, verification is automated where it can be: tests that gate every merge, staged rollouts with health checks, and service-level objectives reviewed regularly so they still reflect what users care about. For people, verify that new engineers can deploy, roll back and run an incident with the real tools, in a rehearsal, before they're on call. For AI coding tools, pair the rollout with review capacity and tests, because more code without more checking is more risk. SWD-REL-06 · SWD-REL-03 · SWD-SLO-03 · SWD-AI-04
For engineering work, verification has a professional dimension: the licence holder stays responsible, and reviewers must be qualified for the work they review. Review engineering work throughout the project rather than once before issue, and check information against the agreed requirements at every stage gate. For new staff, verify that they can interpret what a model or analysis is telling them, not just operate the software. PRM-ENG-01 · SOP-GOV-08 · BM-QA-04 · BM-ROL-04
On site, verification lives at the hold points. Put hold points at irreversible stages, define hold, witness and review points in the contract, release them with the inspector rather than by phone, and track every bypass. Supervisors verify in the field, and a ticked box isn't a verified outcome. Audit one feature from requirement to as-built now and then; it finds what document audits miss. QMS-ITP-02 · QMS-ITP-03 · QMS-ITP-04 · SOP-USE-04 · SOP-GOV-09
In plants, verification gates start-ups and changes. Complete the pre-start-up safety review before hazards are introduced, never start up past known instrument faults, and hand over system by system with operations accepting each. For digital twins and analytics, budget for verification and validation, and demand measured, not projected, savings. TAR-SAF-02 · TAR-SAF-03 · TAR-SU-02 · DM-TWN-05 · FAM-NRG-03
In finance, verify that controls operate, not that they're documented: check the report before you trust it, set thresholds for reviews so they're more than a glance, and watch late, manual and senior entries. Where duties can't be segregated in a small team, supervise instead. FIN-CTL-02 · FIN-CTL-01 · FIN-CTL-05 · FIN-CTL-06
A design-build contractor preparing to move a 50-metre bridge span in one weekend found that its independent design review covered only the finished structure. The connection nodes and the construction stages, where the span would see its largest loads on the transporters, were outside the reviewer's scope and fee. Nobody had done anything wrong; the scope had simply been set by someone who didn't ask where the hazard was. Extending the review, with budget, to the nodes and every move stage cost a small fraction of the contract. It's the kind of gap that only appears when someone asks what each check actually covers, which is the core habit of this stage. DQ-UP-05 · DGN-GAP-02 · SOP-GOV-02
Verify hands Adopt a short, honest record: who has been verified on what, by what kind of evidence, and what isn't verified yet. Include the systems that passed commissioning, the procedures that passed field testing, the checks that were themselves checked, and the gaps. A record that says "seven of nine coordinators verified unaided; two need another supervised job" is far more useful than one that says "training complete". #28 · DGN-CNS-01
Define "done" for verification with a rubric rather than a date. If the rubric isn't met, the honest options are more practice, a narrower scope, or a delay. Moving to Adopt anyway is a decision, and like every override, it should have an owner and a record. IDE-DES-04 · OGV-GATE-01
For small organizations, verification is mostly about two habits: watching the first real job before letting someone work alone, and periodically picking one finished piece of work and checking it against the standard from start to end. Both cost little, and together they catch most of what matters. SOP-USE-06 · SOP-GOV-09
The checklist. Show me, don't tell me. Test unaided, on job-like tasks, and again at 30 to 90 days. Check with AI off. Treat certificates and hours as signals. Rehearse as a team, with surprises. Commission systems and test restores. Test the test; scope review to the hazard; audit the work, not the binder. Verify models, schedules and AI on your own cases. Never start up past an open safety review. Record what isn't verified. Then go to Adopt.
A change is adopted when the new way is simply how work gets done, and the old way has quietly disappeared. That rarely happens on its own. This brief covers staged rollouts, measuring real use, making the new way the easy way, and the people who carry it.
Adopt makes the intended workflow the normal way work is performed. Not an option, not a pilot, not "the new system some teams use": the way. By the time a change reaches this stage, people have learned it, applied it on real work and been verified as able to do it. Adopt scales that from a bounded first use to the whole organization, and removes the old way so nobody slides back. #63 · PM-10
This is the stage where many changes stall without anyone deciding they should. Licences are bought, training is delivered, a launch email goes out, and a year later half the teams still email drawings, keep their own spreadsheets and treat the new platform as an extra step. Nothing failed, exactly; adoption just never finished. The common causes are familiar: the old way was still available and easier, incentives didn't change, managers didn't ask for the new way, and success was measured in logins. CM-DIG-03 · OC-05 · #53
A useful finish line: a change is adopted when the old way is harder than the new way, the people who do the work would object to going back, and the organization can show the new workflow running across the intended scope. The rest of this brief works toward that. #64 · IDE-AEC-02
Design adoption as a socio-technical operating change. A new common data environment, inspection process or finance system changes roles, hand-offs, incentives, procedures and information all at once. If only the technology changes, the surrounding system pulls people back to the old way. In marketing and customer channels, for example, staffing and incentives must follow the new channel, or staff will steer customers back to the one they're rewarded for. The same is true of a site team whose foreman is still judged on the old metrics. #63 · OC-05 · ITG-CHG-01
Treat change as a governed practice embedded across the whole delivery lifecycle, not a bolt-on approval gate at the end. That means the change appears in project plans, budgets, role descriptions, contracts and performance conversations, and it has an owner who stays with it after launch. A change that belongs to a project team that disbands at go-live has no owner during the months when adoption is actually decided. PM-10 · #9 · KIN-GOV-01
Leadership has to be visibly committed, and visible means behaviour, not announcements. People watch whether senior staff use the new workflow themselves, ask for its outputs in meetings, and protect time for it when deadlines bite. They also watch whether senior staff are trained on it: adopting new tools bottom-up works better when senior staff are trained too, in a form that fits their role, because a principal who can't judge work in the new system will ask for it the old way. #7 · LSC-ORG-05 · LSC-AGE-03
Explain the change in the terms of the people living with it. "It supports our digital strategy" persuades nobody on site. "You'll stop re-entering the same inspection data in three places" might. Relevance in the learner's own terms is one of the cheapest adoption tools available, and it should be repeated by local leaders, not only by the project team. #26 · #38
Scale the change in waves, with a check between each. The software world has learned this the hard way: ship in small batches, roll out in stages with automated health checks, and keep the ability to switch a change off. The 2024 CrowdStrike outage, which needed hands-on recovery on millions of machines, is the cautionary case; the company's own analysis says the faulty content should have had a staged deployment. SWD-REL-03 · SWD-REL-01 · SWD-REL-04 · DQ-UP-08
The offices that haven't started yet are your comparison group. Measure the work in both, so you can tell the change's effect from everything else going on. IDE-AEC-02 · IDE-EV-03 · OC-14
Each gate should ask about the work, not the calendar. Is the defined workflow actually running in the current wave? Are the success measures moving? Is the support load falling? Are the champions coping? If a gate fails, fix the issue before the next wave, because problems multiply with every office you add. SEM-30 · #64
Scaling, not technology, is usually the bottleneck. Studies of digital manufacturing keep finding pilots that worked and never spread, because the pilot had special attention, special people and special exceptions that the next site didn't have. Plan the scale-up while you plan the pilot, and judge the pilot on whether it can be repeated without its heroes. DM-ARC-07 · #1
Pair AI rollouts with review capacity. AI coding and drafting tools increase output; without matching review capacity and tests, they also increase the volume of unchecked work. Scale the tool and the checking together. SWD-AI-04 · SWD-AI-05
The most misleading adoption number is the login rate. Ninety-two per cent of staff logging in on day one says almost nothing about whether the new workflow is how work gets done. Measure depth of implementation, not adoption: which parts of the intended workflow are actually running, on which projects, and with what effect on the work. CM-DIG-03 · IDE-AEC-02 · BM-MAT-03
Vendor dashboards usually report levels 1 and 2. Workflow data, such as models issued on time, clash cycles, digital daily reports and close-out time, shows levels 3 to 6. IDE-AEC-02 · CM-DIG-03 · BM-EVD-02
Read every adoption statistic for what it measures. Industry surveys of BIM, AI or platform adoption usually measure perception or intent, not use. Self-assessment scores make poor key performance indicators because people learn to score well. Your own workflow data, collected the same way before and after, is the only adoption measure worth managing to. BM-EVD-02 · BM-MAT-02 · SWD-PLT-04 · DM-AIM-05
Look for the parts that didn't move. At level 3, the easy tasks migrate and the awkward ones stay behind: the exception, the retroactive change, the subconsultant with a different format. Those leftover tasks are where the old way survives, and they usually point to a gap in the defined workflow, the tools or the training. DGN-GAP-04 · #32
People take the easiest route that gets the job done. Adoption succeeds when that route is the new one. A few design choices make the difference:
Remove barriers that make some people's new way harder than others'. Language, devices, connectivity on site, accessibility and working hours all decide whether a mandatory change is genuinely possible for everyone. Remove access and inclusion barriers before making new proficiency mandatory. #40 · GL-18 · IDE-MM-07
Make inefficiency visible. Experienced people who keep the old method often do so because their results look fine. Showing them, on their own recent tasks, how long the old way took compared with the new one is more persuasive than an instruction from above. LSC-SFT-02 · LSC-SFT-01
Champions carry adoption between waves and after the project team leaves. The evidence from health-care technology rollouts favours volunteers over appointees, several per site rather than one, and people who can explain why a workflow works, not just which buttons to press. Give them protected time, connect them with each other, and check whether they're making a difference. #8 · LSC-MNT-05 · #33
Managers carry it in the daily conversation. Equip managers to coach adoption: the questions to ask, the outputs to expect in the new form, the signs that someone is struggling. Supervisor support is among the strongest predictors of whether new skills are used at work, and the manager who still accepts the old spreadsheet undoes the rollout one report at a time. #53 · LSC-TRF-04 · PRM-LRN-04
Expect weak voluntary uptake for anything that isn't linked to the job. When a continuing-education programme for engineers was voluntary, only a minority took part. Link the new way to real work, real recognition and real requirements rather than relying on goodwill. LSC-CPD-06 · #27
Be careful with data about people's adoption. Field crews and unionized teams may see usage telemetry as surveillance, and that suspicion can stall adoption faster than any technical problem. Agree what the data is for, keep it out of performance appraisal, aggregate where you can, and say how long you keep it. IDE-DAT-05 · #48 · DGN-AI-01
Adoption always means the new way becoming normal. What "normal" looks like, and what keeps the old way alive, differs by team. #63 · CM-DIG-03
For an internal platform, treat adoption as product work: build golden paths that make the right way the easy way, measure developer experience including cognitive load, and watch for the platform adding friction rather than removing it. Platforms remove toil, not ownership; a team that can no longer understand its own deployment has adopted a dependency, not a capability. Read vendor adoption forecasts as forecasts. SWD-PLT-01 · SWD-PLT-05 · SWD-PLT-03 · SWD-PLT-02 · SWD-PLT-04
BIM adoption has repeatedly plateaued at shallow levels: models produced, but not used for coordination, quantities or handover. Read plateaus as a depth problem, choose model uses from the owner's goals, starting with the downstream uses, and measure coordination value yourselves rather than quoting a percentage. Adoption works better bottom-up, on live work, with senior staff trained too. BM-MAT-03 · BM-USE-01 · BM-COO-05 · LSC-ORG-05
On site, the old way survives on paper. A new digital daily report won't be adopted while the paper one is still accepted, and a new inspection plan won't be followed while the old checklist is still in the site office. Retire the old forms, keep one current version at the point of use, and make the tools work on site devices with poor connectivity. SOP-DIG-01 · SOP-DIG-03 · FAM-SYS-04
In plants, adoption stalls when the system doesn't explain itself and operators don't trust it. Co-design with operators, keep a person on the edge cases, and scale from the pilot line only when the pilot can run without its heroes. Expect the gains of industrial AI to arrive after a dip, not before it. DM-WRK-02 · DM-AIM-03 · DM-ARC-07 · DM-AIM-01
For customer channels, adoption depends on staffing and incentives following the new channel, one source of truth for what can be promised, and proving one journey before rolling out the rest. Don't try to be everywhere at once. OC-05 · OC-03 · OC-14 · OC-02
An eight-office engineering firm rolled out a new common data environment. Day one looked like a triumph: 92% of staff logged in. Three months later, half the offices were still sending models by email, because the subconsultants they worked with weren't on the platform and the email route was faster. The fix wasn't more training. It was a contract change requiring subconsultants to deliver through the platform, a read-only switch on the old shared drive, and two volunteer champions per office with protected time to help the stragglers. Measured by models issued through the platform with recorded review, adoption went from about half to nearly all within two months. CM-DIG-03 · #62 · LSC-MNT-05 · IDE-AEC-02
In construction and asset projects, adoption has a second meaning: the people who will operate the asset have to adopt the information the project produced. Make handover information contractual before buying a twin, require structured asset data during construction rather than at the end, engage facility managers from concept, and run a handover period in which operations formally accepts the information. Native formats inside the team are fine; open formats at every contractual boundary keep the information usable after the project is gone. CM-DIG-02 · FAM-DAT-02 · BM-USE-05 · FAM-DAT-04 · BM-EMG-05
Across firms, adoption needs alignment as much as inside one. Align partners on digital goals, roles and success conditions, and check against the asset information requirements at every stage gate, not only at the end. #62 · BM-QA-04
Adopt hands Support an organization where the new way is normal across the intended scope, with a known list of the places it isn't yet and why. Hand over the champion network, the help content, the question log from Apply and the depth measures, because Support will use all four. Plan who hears what about the result and when; the announcement that a change is "complete" can itself signal to people that attention has moved on. DGN-CNS-04 · #30 · KIN-GOV-03
The checklist. Change roles, incentives and procedures with the tool. Give the change an owner after go-live. Train senior staff too. Roll out in waves with gates about the work. Measure depth, not logins. Close the old route. Make it work on site and on mobile. Put volunteer champions and coaching managers in every wave. Keep usage data out of appraisals. Make handover information contractual. Then go to Support.
After a change is adopted, most learning happens in the middle of a task, when someone hits the case nobody trained them for. Support is the system that answers them quickly, from the right source, and learns from the questions.
Support provides searchable, contextual help and human support at the moment of need. It begins in earnest once a change is adopted and runs for as long as the work does. Training prepares people for the common cases; Support catches everything else: the rare task done twice a year, the exception the procedure didn't anticipate, the software update that moved a button, the new starter who joined after the rollout. #22 · IDE-TEC-05
Support is also the stage most often left unfunded. Budgets cover the build and the launch; the months afterwards, when people are actually using the new way and questions are most frequent, often have no owner and no money. That's when people quietly find their own workarounds, the defined workflow drifts, and the gains measured at launch start to fade. Until the Database's stage audit in 2026, this was also the thinnest stage in its own advice: most practices were written for launch, not for the long life afterwards. #64 · #21 · #57
A useful finish line, which you never quite cross: someone stuck in the middle of a task can find a trustworthy answer in under two minutes, from inside the tools they're using, and the organization learns something from every question that couldn't be answered.
Performance-support researchers describe several distinct moments when people need help, and each calls for a different kind of support. Designing for only the first, learning something new, is why so many organizations have a training catalogue and still have people stuck. IDE-TEC-05 · #22
Moments 3 to 5 are the Support stage. They happen during the work, so the help has to live there too. The framework comes from performance-support practice and is mostly case evidence. IDE-TEC-05 · #21 · LSC-MOD-02
The distinction matters because the right format differs. A course is the wrong answer to moment 3: nobody leaves a half-finished model or an open isolation permit to take a module. A one-page aid is the wrong answer to moment 1: people learning something new need examples, practice and feedback. Most support failures are format mismatches, not content gaps. LSC-DES-04 · #22
Train complexity; deliver specificity. The reasoning behind a method belongs in training, because it builds judgment that lasts. The click-by-click steps, the form fields and the settings belong in support, because they change and nobody should memorize them. LSC-DES-04 · #21
Put performance support inside the systems people already use: the common data environment, the maintenance system, the ERP, the CRM, the design tools. For rare or high-consequence tasks, short, findable guidance at the step is far more useful than a library people have to remember to visit. "Learning in the flow of work" is a newer name for this older idea; the principle hasn't changed. IDE-TEC-05 · #22
A few design rules make in-tool help actually used:
Govern the content as carefully as you'd govern a procedure. Curation needs an owner, pathways need quality checks, and old material needs retiring. A support library that grows without anyone removing anything becomes the problem it was meant to solve. #57 · KIN-GOV-01 · KIN-GOV-02
Capture knowledge in the flow of work, too. When an experienced technician solves an unusual fault, a two-minute recording or a short note added to the asset's record is worth more than any later documentation project. Let knowledge travel with the asset, so the next person who works on that pump finds what the last one learned. KIN-CAP-02 · KIN-TWN-03 · FAM-SYS-05
Documents answer the questions someone anticipated. People answer the rest. Most organizations already have an informal support network, the colleague everyone asks; Support makes that network deliberate, resourced and visible. LSC-MNT-03 · #33
Each tier should know when to pass a question up, and each question that reaches tier 3 or 4 is a hint that tier 1 is missing something. #64 · #32
Maintain a community of practice for fast-moving tools. Where software and methods change monthly, a regular forum where practitioners share what they've found keeps knowledge moving faster than any formal update. Champions anchor it locally; give them protected time, more than one per site, and evaluate whether they're helping. #33 · #8 · LSC-MNT-05
Equip managers to coach. Supervisors are the support most people actually see. A manager who asks how the new process is going, notices who is struggling and makes time for practice does more for long-term use than any help desk. Training sticks when the workplace backs it. #53 · SOP-TRN-05 · LSC-TRF-04
Mentoring is support with a longer horizon. Its effects are small but real. Informal mentoring has looked stronger than formal schemes, but people who find their own mentors differ in motivation and access, and a purely informal system leaves some people with nobody. Keep a formal route open and give it informal qualities: choice in the match, time in the mentor's workload, and feedback on real work. LSC-MNT-01 · LSC-MNT-02 · FAM-WRK-02
Protect the help desk itself. It's a common target: attackers impersonate staff to get passwords and multi-factor authentication reset. Harden identity checks for resets, and make sure every critical role has a backup so support doesn't stop when one person is away. CYBER-05 · FIN-TAL-07
Support covers the systems, not just the people. Once a new workflow depends on a platform, an analytics feed or an AI tool, someone has to own its behaviour in operation. AIG-01 · DM-AIM-03
For operational systems, alert on what users experience, not on every internal cause; an alert nobody acts on trains people to ignore alerts. Analytics only pay when someone acts on the findings. Building fault-detection software that flags a hundred faults a week and changes nothing is a support failure, not an analytics one. SWD-SLO-05 · FAM-NRG-02 · FAM-NRG-01
Plan support capacity for bad days. During incidents and long events, staff around rest and handovers rather than heroics, and set service tiers in advance so everyone knows what stops first when capacity runs short. DQ-RT-08 · HX-09 · SEM-09
Every support request is data. Collected and read, questions are one of the best diagnostic signals an organization has, because they show exactly where the defined workflow, the training or the tools fall short, in the words of the people doing the work. #32 · KIN-SIG-01
| Signal | What it usually means | Next step |
|---|---|---|
| The same question from many people | A gap in the aid or the training | Fix the aid; add it to the next refresher |
| Searches that return nothing useful | Content missing, or named differently from how people think | Write it; use their words |
| A workaround that keeps appearing | The workflow doesn't fit the work | Send to the procedure owner (Adapt) |
| Low use of the learning on offer | Relevance, access or time, not lack of content | Diagnose before adding more |
| Questions stop entirely | Mastery, or people have given up asking | Check which, before celebrating |
Treat friction as a sensor and measure knowledge flow, not uploads. KIN-SIG-01 · KIN-SIG-02 · #15
Diagnose low participation before adding content. When people don't use the support or learning on offer, the instinct is to produce more of it. The cause is usually elsewhere: it isn't relevant to their work, it's hard to reach from site, or nobody has time. More content solves none of those. #15 · #40 · LSC-ORG-02
Treat deviations the same way. A shortcut that people take routinely is a support signal: either the procedure is wrong, or the help that would make the right way easy is missing. Route those signals to the people who own the workflow, and close the loop so the person who raised it sees something change. SOP-USE-05 · SOP-USE-01 · SOP-GOV-06
Be careful how you collect usage data. Agree its purpose with the people it describes, keep it out of performance appraisal, and tell people what's recorded and why. Support that feels like surveillance stops being asked. IDE-DAT-05 · IDE-DAT-02 · #48
The moments of need are universal; what helps at each one depends on the work. IDE-TEC-05 · #22
For digital services, support includes the people running them. Alert on what users experience, give incident communication its own owner, and close the learning loop after incidents deliberately rather than hoping someone remembers. Internal platforms should be treated as products, with paths that make the supported way the easy way and someone who listens to the teams using them. SWD-SLO-05 · SWD-INC-04 · SWD-INC-07 · SWD-PLT-01
For design teams, the most valuable support is usually a person: the information manager who knows why a delivery failed the checks, the senior engineer who will look at a strange analysis result. Keep one authoritative container per type of information, use the common data environment's states as intended, and capture the reasoning behind decisions where the next team will find it. A model holds information; people hold the knowledge around it. BM-CDE-03 · SOP-DIG-02 · KIN-FND-01
On site, support has to work offline, on a phone, in a noisy place. Keep version-stamped aids at the point of use, use QR codes that lead to the current version, and give multi-employer sites their own coordination procedure. Supervisors are the support crews see most; give them the one-page guides and the time to use them. SOP-DIG-03 · SOP-USE-07 · SOP-USE-04
In plants and buildings, support is tied to assets. Keep one authoritative asset register, let maintenance events update the records automatically, and let knowledge travel with the asset so the next technician sees the last one's notes. Broker vendor remote access through jump hosts with strong authentication, and patch by consequence rather than by calendar. FAM-SYS-02 · FAM-SYS-05 · KIN-TWN-03 · DM-SEC-04 · DM-SEC-05
Some support has deadlines in law. Requests from people to see, correct or erase their data must usually be answered within a month, and an access request means all of their data, redacted where necessary rather than refused. Know where personal data lives before the first request arrives, and keep any automated decision with a human route. GDPR-14 · GDPR-15 · GDPR-16 · GDPR-18
A facilities team introduced a new maintenance system across forty buildings. Training went well; three months later, technicians were still phoning the same senior colleague about the same six tasks. Rather than rerunning the course, the team turned those six calls into six one-page aids attached to the relevant asset types in the system, and gave the senior colleague two hours a week, protected, to answer whatever was left. Calls on those tasks dropped sharply, and the question log started showing new problems instead of old ones, which is what a working support system should do. #32 · #22 · FAM-SYS-05 · KIN-CAP-02
Synchronize content with software releases. In fast-moving tools, a job aid can be wrong within weeks of being written. Tie content updates to the release calendar, review procedures on triggers such as incidents, changes and new regulations, with a calendar date as a backstop, and retire what's out of date rather than leaving it to be found. #21 · SOP-GOV-01 · SOP-GOV-07
Keep the information that support depends on. One authoritative asset register and space hierarchy, maintenance events that update the records automatically, and your own copy of building and system performance data protect support from vendor changes and staff turnover. Treat information continuity as a duty, even where it isn't the law. FAM-SYS-02 · FAM-SYS-05 · FAM-NRG-04 · FAM-DAT-05
Support hands Measure two things. The first is its own data: questions, searches, escalations and response times, which show where the change is working and where it isn't. The second is a set of patterns worth testing: if questions about one task dropped after a new aid went live, Measure can check whether the work improved too. Support and Measure run side by side for the life of the change. #64 · #10
For small organizations, Support can start with three things: a shared folder of one-page aids with an owner, a named go-to person for each system with time set aside, and a running list of questions reviewed once a month. That's a support system, and it beats a help portal nobody maintains. #30 · #33 · #32
The checklist. Design for moments 3 to 5: applying, failing and changing. Put help inside the tools, at the step. One discovery point, one owned version. Build a community of practice with resourced champions. Equip managers to coach. Own every AI output and keep a way to pause it. Log and read the questions; treat friction as a sensor. Diagnose low use before adding content. Sync content with releases. Then go to Measure.
Measure tracks what changed in learning, behaviour, the workflow, the business and the workforce, and separates the change's effect from everything else going on. Most organizations measure what's easy. This brief is about measuring what's true.
Measure tracks learning, behaviour, workflow, business and workforce outcomes, and asks how much of any change was caused by what you did. It was set up in Define, when the success conditions and baseline were written down; it runs alongside Support for as long as the change matters; and it feeds Adapt, which decides what to change next. #10 · #14 · IDE-DES-07
Measurement goes wrong in two predictable ways. The first is measuring what's easy instead of what matters: satisfaction scores, completions, logins, hours logged. These are cheap to collect and almost never answer the question leaders actually ask, which is whether the work got better. The second is measuring the right thing badly: a before-and-after comparison that ignores everything else that changed, a baseline taken at the worst month, or a return on investment built on assumptions nobody would defend. LSC-MSR-01 · DGN-BSL-01 · IDE-EV-12
A useful standard: a measurement is good enough when a sceptical finance director could read how it was done and agree with the conclusion, or see exactly where the uncertainty lies. That's achievable for most organizations with records they already keep. It rarely needs a new platform; it needs a plan made before the change starts. LSC-ORG-03 · IDE-EV-08
Evaluation is usually described in levels: how people reacted, what they learned, whether they apply it at work, what changed in the business, and whether the return justified the cost. Each level answers a different question, costs more to measure than the one before, and moves on a longer timescale. Not every programme needs every level. Set evaluation targets by level, and choose the deeper levels for programmes that are strategic, expensive or highly visible. IDE-EV-01 · #10
Targets from published evaluation methodology, not research findings. The learning-level target rose over time; the deeper levels stayed about the same. IDE-EV-01 · IDE-EV-10
At the reaction level, ask the right question. How much people enjoyed training barely predicts anything; how useful they judged it predicts on-the-job use better than test scores do. Replace "how did you like it?" with "how are you using this in your current work?" LSC-MSR-02 · LSC-MSR-01
At the learning level, test performance on new, job-like tasks without aids, and test again after 30 to 90 days. Immediate tests mostly capture what people can do while the training is fresh. LSC-MSR-03 · LSC-MSR-04
At the application level, measure once the new behaviour should be routine, often three to six months after training rather than a fixed few weeks. Measure behaviour separately from knowledge: people can know the right answer and still not do it. IDE-EV-02 · GL-14
At the impact level, use the records you already keep. For technical work, that means quality defects, design-attributable RFIs, rework hours, time to a first independent deliverable, near-miss reports and close-out time. For digital adoption, measure the workflow itself: models issued on time, clash cycles, digital daily reports. LSC-ORG-03 · IDE-AEC-02 · QMS-COQ-03
The hardest part of measurement is not collecting numbers; it's knowing what caused them. Projects rarely run in a vacuum: a new manager arrives, the market shifts, a difficult client finishes, the season changes. A before-and-after comparison can't separate the change's effect from all of that. IDE-EV-03 · LSC-MSR-05
Isolate the effect with a comparison group first. The best comparison is often free: a staggered rollout, where the teams or offices that haven't started yet act as the comparison for those that have. Where no comparison is possible, use trend lines or forecasts, and only then fall back on estimates from participants or managers, adjusted for how confident they say they are. IDE-EV-03 · LSC-MSR-05 · IDE-AEC-02
Get the baseline right. Never baseline at the moment the problem was noticed, because problems get noticed at their worst and tend to drift back toward normal on their own, which flatters any fix. Use several measurements from before the change, covering at least one full operating cycle, and say why the window was chosen. DGN-BSL-01 · DGN-BSL-04 · QMS-LSS-05
Prefer objective records to self-reports. Self-rated confidence inflates results, and so do self-assessment scores used as key performance indicators: people learn to score well. Where you must survey, read the results as perception data, and check survey timing against recent events. LSC-MSR-05 · BM-MAT-02 · BM-EVD-02 · DGN-DATA-01
Measure speed from telemetry, not opinion. When people are asked how much faster an AI tool makes them, the answer and the measurement often disagree, and in one controlled study experienced developers on familiar code were slower with AI assistance while believing they were faster. Measure the speed-up; don't ask about it. PRM-EVD-08 · SWD-AI-03 · SWD-AI-02
Some indicators are so convenient that they survive long after their weaknesses are known. Learn to recognize them, and keep them out of the headline. SOP-GOV-10 · CM-EVD-01
| Indicator | What it really measures | Use instead |
|---|---|---|
| Satisfaction 4.7 out of 5 | How pleasant the course felt | Unaided performance on a job-like task, later |
| Logins or licences active | Access, not use | Workflow steps running the new way |
| Recordable injury rate | Mostly minor, reported injuries | Leading indicators and major-hazard precursors |
| Percent complete by hours spent | Effort consumed | Physical progress against rules agreed in advance |
| Badge swipes | Attendance | Sensor or observation data on actual use |
| CPD hours logged | Compliance | Competence shown on the work |
| Projected savings | A model's prediction | Measured savings, with measurement and verification |
None of these is useless; each is fine in its own lane. The harm comes from reading one as the other. LSC-MSR-01 · CM-DIG-03 · SOP-GOV-04 · PCF-EVM-01 · FAM-SPC-01 · LSC-CPD-01 · FAM-TWN-06
Two cautions about safety measures in particular. First, retire the recordable injury rate as the headline safety measure, and track leading indicators alongside it. Second, keep personal-safety and process-safety indicators separate: at Texas City, a very low injury rate gave a misleading picture of the risk of a major accident. Correct injury counts for under-reporting before trending them, especially where anyone's bonus depends on the count. SOP-GOV-04 · CM-SAF-02 · DGN-BSL-02 · DGN-BSL-03
Don't turn measures into targets or league tables. Delivery metrics in software, for example, are useful for a team to understand itself and harmful when used to rank teams, because people optimize the number rather than the work. Track speed and stability separately, and use several measures of productivity rather than one. SWD-MET-04 · SWD-MET-05 · SWD-MET-06 · SWD-MET-02
Measure quality locally, with definitions people agree on, and read every number in context. A defect rate that rose because inspection improved is good news that looks bad. QMS-ARC-04 · SOP-GOV-10 · DM-STD-04
Leaders increasingly ask for the financial return on training and change, and most organizations can't answer. In one survey, nearly all chief executives wanted to see the business impact of training, but fewer than 1 in 10 received it. The usual barriers are isolating the effect and getting the data, which is why the data connections should be set up before leaders ask. IDE-EV-11 · IDE-EV-10
When return on investment is calculated, it's often inflated. Very high published figures usually come from counting several years of benefit, using undiscounted estimates, counting revenue instead of margin, or treating freed time as cash. Applying conservative rules to one widely circulated example turned a claimed return of about 540% into about 70%. IDE-EV-12 · IDE-EV-04
Seventy per cent in a year is still a good return. The problem with 540% isn't that it's too high; it's that nobody could defend how it was reached. IDE-EV-04 · IDE-EV-07 · IDE-EV-08
A defensible financial evaluation follows a few rules:
For complex programmes, such as leadership development, the Success Case Method is often cheaper and more convincing than a full return calculation: study the people for whom the programme worked best and worst, and find what made the difference. IDE-EV-09
Measurement covers forecasts and controls too, and both are easier to fool yourself about than training results. Score forecasts and feed the results back. When estimators record a probability for a milestone, a cost or a risk, scoring it once the outcome is known shows who is calibrated, not just who was right. Used to punish, scoring makes people hedge toward 50%; used for feedback, it improves forecasting. Measure who is optimistic, too: in finance teams, a forecaster whose numbers are always high is valuable information. DQ-LRN-01 · FIN-FPA-02 · DQ-LRN-05
On projects, measure progress physically, with rules agreed in advance, never by hours or money spent. Treat an early bad cost index as a warning that rarely improves much later; the forecast it implies is a floor, not a ceiling. Keep an independent forecast beside the team's, and report forecasts as ranges with a stated confidence. PCF-EVM-01 · PCF-EVM-02 · PCF-EVM-04 · PCF-FC-01 · PCF-FC-03
For controls and governance, measure whether controls operate, not whether policies exist. A policy that says access is reviewed quarterly is not evidence that it was. For savings claims from energy, maintenance or digital-twin projects, require measurement and verification: projected savings are not measured savings. AIG-16 · FAM-NRG-03 · FAM-TWN-06
Use the right tool for repetitive measurements. Statistical process control charts suit repetitive, frequently measured processes, such as a production line, but not one-off construction activities; expect false alarms, and more of them as you add rules. QMS-SPC-01 · QMS-SPC-03
Most teams already collect the data they need. The skill is choosing the measures that reflect the work and reading them honestly. LSC-ORG-03 · SOP-GOV-10
Use the current delivery metrics, labelled with the year their definitions come from, and measure them from your own systems rather than surveys. Read published performance tiers as descriptions, not benchmarks, and don't turn the metrics into team rankings. Retire the mean time to restore as a headline metric; it's dominated by a few long incidents. Code health is worth tracking, because it predicts defects and delivery speed. SWD-MET-01 · SWD-MET-03 · SWD-MET-02 · SWD-INC-08 · SWD-TD-03
On projects, measure physical progress, Percent Plan Complete as an early warning, leading safety indicators, rework and the cost of your own nonconformances. Measure coordination value yourselves, by tracing resolved clashes to the RFIs and change orders they prevented, and measure contract value loss from your own ledger rather than an industry figure. PCF-EVM-01 · CM-LEAN-02 · CM-SAF-02 · QMS-COQ-03 · BM-COO-05 · VCM-ADM-03
In manufacturing, define equipment effectiveness and other key indicators the standard way, capture machine state automatically, and pair effectiveness with throughput so one isn't improved at the other's expense. In buildings, track a few maintenance indicators against simple targets, define a space-metric stack before comparing utilization figures, and remember that badge swipes are attendance, not use. DM-STD-04 · DM-SUS-03 · FAM-MNT-03 · FAM-SPC-02 · FAM-SPC-01
For training, measure performance on new, job-like tasks, re-test after 30 to 90 days, use comparison groups and objective records, and treat licensure hours as a compliance floor. For engineering firms, the records that already exist, such as quality defects, design RFIs, rework hours and time to first independent deliverable, are better outcome measures than any survey. LSC-MSR-03 · LSC-MSR-04 · LSC-MSR-05 · LSC-CPD-01 · LSC-ORG-03
For AI tools, test prompt and model variants on many real examples, calibrate any AI-as-judge against people, measure speed-ups with telemetry, and know which adoption figure you're reading: "uses AI weekly" and "relies on AI for a core process" are very different numbers. PRM-EVD-04 · PRM-VER-06 · SWD-AI-03 · DM-AIM-05
A mechanical contractor's new onboarding programme scored 4.7 out of 5 for satisfaction. The finance director asked whether apprentices could actually do the work. The training lead compared weld reject rates, spool rework and unaided performance on a new spool at day 90 for three groups: last year's cohort, this year's, and a small group that started later and was trained the old way. This year's cohort did noticeably better on all three, and the report said plainly that the groups were small and the results encouraging rather than conclusive. That honest framing is what made the board fund the programme again. LSC-MSR-05 · LSC-MSR-03 · IDE-EV-07
Measure hands Adapt evidence: what changed, at which levels, compared with what, and how sure you are. It should include what didn't change and what you couldn't measure, because those are often where Adapt's best questions come from. DGN-CNS-01 · #64
Run and share your own evaluations. Engineering, construction and manufacturing have very little controlled evidence about their own training and adoption programmes. A well-designed evaluation, shared with a trade association or professional body with its limits stated, adds to what the whole sector knows. LSC-MSR-06 · LSC-CPD-05
Feed the results into decisions, not reports. Measure IT performance and business benefits separately, and feed both into the next portfolio cycle; track benefits against the owners named in Define. A measurement that never changes a decision wasn't worth collecting. ITG-VAL-01 · ITG-VAL-02 · OGV-BIZ-02
For small organizations, a credible evaluation fits on a page: one business measure from records you already keep, a baseline from several months before, the teams that started later as a comparison, a check at three months, and an honest sentence about what else might explain the result. LSC-ORG-03 · IDE-EV-03
The checklist. Plan the measurement in Define. Evaluate by level, deeper for what matters most. Ask about usefulness, not enjoyment. Test unaided, later. Use a comparison group, often a staggered rollout. Never baseline at the worst month. Prefer records and telemetry to opinion. Keep satisfaction, logins and injury rates out of the headline. Report money as a first-year range with the method shown. Score forecasts; measure that controls operate. Share what you learn. Then go to Adapt.
Adapt uses everything the earlier stages produced, the evidence, the questions, the failures and the results, to change the system itself. It's the stage that turns a one-off project into an organization that gets better at change.
Adapt uses evidence, questions, failures and outcomes to change the system and restart the cycle. It's where the measurements from Measure, the questions from Support, the deviations from Apply and the gaps from Verify become decisions: change the procedure, fix the template, retire the course, rebuild the pathway, or stop the project. Then the next diagnosis begins, better informed than the last. #64 · KIN-CAP-03
Most organizations are good at writing lessons down and poor at changing anything because of them. A lessons-learned register fills up, the next project repeats the same mistakes, and the reviews start to feel like ceremony. Public inquiries see this at national scale: the Grenfell Inquiry recommended a statutory public record of inquiry recommendations and the government's response, precisely because so many earlier recommendations were never acted on. The test of Adapt is not how many lessons were recorded, but how many things changed. DQ-LRN-04 · SOP-GOV-06 · WF-GOV-02
A useful finish line: every significant lesson has an owner, a change it led to, and evidence that the change worked, and the next cycle starts from a better baseline than this one did.
Learning starts with an honest account of what happened, which depends on people being willing to give one. Run reviews under a just culture: blame-aware, separating the question of what happened and why from any question of individual culpability, by procedure and by people. Protect psychological safety, so that failure analysis is candid. In environments where admitting a mistake costs a career, reviews produce tidy stories instead of useful ones. SWD-INC-05 · CYBER-27 · DGN-RCA-03 · CS-06
Don't wait for the end. Run after-action reviews during the project, at natural milestones, and reset psychological safety before deadlines, not only at kickoff. A short review after each phase, asking what we expected, what happened, why the difference, and what we'll do next, catches problems while they can still be fixed. After events and incidents, debrief straight away with everyone involved, including other agencies or partners. PM-23 · SEM-33 · HX-18
Three disciplines keep reviews honest:
When consequences are delayed, reconstruct the causal chain explicitly. People learn poorly from outcomes that arrive months after the decision, because the link is invisible. Laying out the chain, decision by decision, is how a debrief makes that link visible again. GL-07 · GL-02
A lesson counts when something changes. Every lesson should end in one of three things: a change to the system, an explicit decision not to change with the reason recorded, or a test to find out more. "Lessons noted" is not one of the options. SOP-GOV-06 · SEM-34
Separate the correction from the corrective action, define the effectiveness check before closing either, and tier the depth of analysis by risk. QMS-CAP-01 · QMS-CAP-02 · QMS-CAP-04
Prefer structural fixes to procedural ones for chronic problems. A new rule, a reminder or another layer of approval is the easiest response to a recurring failure and usually the weakest. Changing the template, the tool, the default, the incentive or the workflow so the error is hard to make tends to work better and last longer. PM-3 · PM-7
Track every recommendation to closure, in front of the people who can make it happen. Keep one register, with an owner, a deadline and evidence of closure for each item, reviewed at the level that has the authority to act: the board for major reviews, the project leadership for project lessons. Measure what changed on the ground, not only what's marked "implemented". DQ-LRN-04 · WF-GOV-02 · WF-GOV-03
Don't close a corrective action on the evidence that it was carried out. Close it on evidence that it worked, checked after enough time has passed to tell. Have management review ask the same question about the whole set: did the actions we closed last quarter actually fix anything? Then re-test: an intervention to reduce noise or bias in judgments is only shown to work when the next audit shows less scatter. QMS-CAP-03 · QMS-ARC-05 · DQ-LRN-05
Changes made in Adapt are changes like any other, and they carry the same risks. Classify every change by its consequence, and check what it touches before making it. A field change is a design change and needs the same review. A temporary change is still a change. In plants, controller changes go through compile, simulation, review, factory testing and management of change, however urgent they feel. SOP-GOV-07 · SOP-GOV-03 · TAR-SAF-04 · PRM-MFG-02
When the evidence shows a procedure that's followed exactly and still fails, redesign it; don't retrain people to follow it harder. Compare the written procedure with the work as it's actually done before deciding which one is wrong, and favour fewer, better procedures over more. SOP-GOV-11 · DGN-GAP-01 · SOP-DES-07
Re-test controls whenever the systems around them change. A financial control that worked in the old system may not exist in the new one; a safety interlock that worked before a controller upgrade needs proving again. If a control now relies on AI, the AI needs evidence too. FIN-TEC-04 · FIN-TEC-01 · TAR-SAF-03
Keep baselines under control. When a project's plan changes, re-baseline formally, with the change recorded and approved, rather than quietly moving dates. A schedule that's updated by moving dates hides exactly the information Adapt needs. When scope is cut, re-run the benefits and readiness diagnosis, because the removed piece may be the one the benefits depended on. PCF-CHG-01 · PCF-SCH-05 · DGN-RDY-04
Keep learning content in step with the system. When a procedure, standard or tool changes, the training, job aids and refreshers built on the old version are now wrong. Synchronize content with releases, and train people before a changed procedure takes effect. #21 · SOP-TRN-07
Sometimes the evidence says the change itself isn't working. This is the hardest call in Adapt, because escalation of commitment is strong and predictable: sunk costs, personal responsibility for the original decision and closeness to completion all push people to keep going. Generic warnings about escalation don't help. Specific decision rules, agreed in advance, do. OGV-ESC-01 · PM-4 · PM-5
Make early bad news cheap and stopping honourable, or people will keep both hidden. OGV-ESC-02 · DQ-LRN-07 · DQ-RT-14 · PCF-RPT-02 · OGV-ESC-03
When a large project resets, have the new estimate checked independently and set it beside the cost of cancelling and the realistic alternatives. Stopping is not cheaper by default, and continuing is not safer by default; the comparison is the decision. DQ-LRN-07 · OGV-PORT-01
Make stopping honourable. A culture in which cancelling a failing project ends careers guarantees that failing projects continue. Recognize the person who raised the flag, and treat a well-reasoned stop as a success of governance. Expect that most projects end "challenged but survived" rather than cleanly succeeding or failing; Adapt is how you make the most of that common middle. OGV-ESC-03 · PM-15
Expect the unexpected inside the plan, too. In plant turnarounds, inspection routinely finds work nobody planned; the turnarounds that cope best plan for discovered work with an assessment process and a contingency, instead of treating every discovery as a crisis. TAR-EXE-02 · PCF-CHG-02
The most valuable adaptations travel. A fix that stays on one project helps one project; a fix that reaches the organization's standards, templates and training helps every project after it. Run a project-to-enterprise learning loop: a deliberate route by which lessons from projects are reviewed, generalized and built into the enterprise's procedures and pathways. KIN-CAP-03 · KIN-CAP-01 · SOP-SYS-05
Treat organizational memory as the result of learning, not an archive. A database of past lessons that nobody reads is not memory. Memory is the set of things the organization now does differently because of what it learned, and the people who know why. Check knowledge readiness at the start of each new project: what did the last similar project learn, and has anyone read it? KIN-CAP-01 · KIN-CAP-04
Use the signals Support collects. Repeated questions, failed searches and routine workarounds show where the system needs to change, often before any metric moves. Treat friction as a sensor, and use knowledge-search behaviour to diagnose gaps. KIN-SIG-01 · #32 · #15
Review the strategy as conditions change. Technology, regulation and the market move; an IT or learning strategy written three years ago may be answering yesterday's questions. Feed results into the next portfolio cycle, so that what you learned changes what you fund. ITG-STR-04 · ITG-VAL-02
Keep the people side adaptive too. Run adoption as a learning loop rather than a fixed plan, keep upskilling existing staff as the work changes, and keep AI-generated learning recommendations explainable and contestable, so people can challenge them when they're wrong. #64 · #5 · #50
Every team runs some version of the learning loop. The difference between teams that improve and teams that repeat themselves is whether the loop ends in a change. #64 · SOP-GOV-06
Software teams have the most mature habits here: blame-aware post-incident reviews, multi-causal analysis and deliberate follow-through. Close the loop by tracking review actions like any other work, and pay down technical debt where the interest is, in the code that changes most often, rather than wherever it's most visible. SWD-INC-05 · SWD-INC-06 · SWD-INC-07 · SWD-TD-02
On projects, feed the reasons for missed commitments into the risk register, run after-action reviews at milestones, and re-baseline formally when the plan genuinely changes. Carry lessons into the next project's mobilization, not just into a register: if the last three projects lost weeks to late subconsultant appointments, change the procurement template. CM-LEAN-03 · PM-23 · PCF-CHG-01 · KIN-CAP-04
In plants, adapting runs through management of change, corrective actions with effectiveness checks, and the condition-based renewal plan. Treat recurring near-identical events as a precursor pattern rather than a series of one-offs, and harden the parts of the operation that the last incident exposed. SOP-GOV-07 · QMS-CAP-02 · FAM-MNT-04 · DGN-RCA-04 · DM-SEC-06
After an emergency or exercise, debrief with every agency straight away, turn lessons into changes and test them, and plan recovery and mental-health support as part of the response rather than after it. Track the recommendations of reviews with owners and deadlines, publish reviews promptly, and measure what changed on the ground. SEM-33 · SEM-34 · WF-REC-05 · WF-GOV-02 · WF-GOV-04 · WF-GOV-03
For learning programmes, each cohort's results should change the next cohort's programme: what to drop, what to move earlier, what new people kept getting wrong. Study the people for whom the programme worked best and worst, and use what you find; keep upskilling existing staff as the work changes; and keep AI-based learning recommendations explainable, so people can challenge them. #64 · IDE-EV-09 · #5 · #50
After a crisis in which nobody was hurt, a contractor's board was ready to record that everything had gone right. A process review, done against what was known at the time, found two calls that had been sound and two that had been lucky: a crack log from months earlier marked "cosmetic, no action", and a stop-work card closed by a phone call. Both were investigated like accidents, traced to the same design detail, and that detail was revised for the next bridge. The board also started a decision log, so that the next review would be able to see what people believed when they decided. DQ-LRN-03 · DQ-LRN-06 · DQ-LRN-08
Adapt ends where Diagnose begins. The evidence gathered here becomes the starting point for the next diagnosis: a sharper gap statement, a better baseline, fewer untested assumptions and a clearer picture of which cause families matter in this organization. Each cycle should be faster and cheaper than the last, because the organization has learned how to change. PM-1 · DGN-DATA-02
Real changes rarely move through the stages in strict order; Support and Measure run in parallel, and a failed gate can send you back to Prepare. The cycle is a map, not a script. PM-20 · #64
For small organizations, Adapt is a habit more than a process: a short review after each significant piece of work, one list of changes with names against them, a monthly look at whether last month's changes worked, and the willingness to stop what isn't working. Those four habits, kept up for a year, change an organization more than any transformation programme. PM-23 · DQ-LRN-04
The checklist. Review under a just culture, during the project as well as after. Judge process apart from outcome. Look for several causes. Investigate near misses. End every lesson in a change, a recorded decision not to, or a test. Prefer structural fixes. Track recommendations to closure where the authority sits, and check the fix worked. Treat your own changes as changes. Agree stop rules, compare stopping with finishing, and make stopping honourable. Carry lessons from projects into the enterprise. Then start again at Diagnose.
Onboarding is the shortest path between hiring someone and trusting them with real work. Most organizations treat it as paperwork and orientation. This guide treats it as a capability programme with a finish line you can check.
Picture three people starting on the same Monday. A site engineer joins a contractor mid-project. A BIM coordinator moves from another firm, fluent in a different authoring tool. A maintenance technician starts at a plant that is replacing half its controllers this year. All three get a laptop, a safety video, a binder and a tour. Six weeks later, one is doing real work, one is quietly redoing everything the old way, and one is waiting for someone to tell them what to do.
The pattern is familiar because onboarding is usually designed around the organization's administration rather than the job. Forms get completed; capability doesn't get checked. The most common failure is that nobody defines what "ready" means, so nobody can tell whether a new person has got there. The second is that the knowledge the person most needs lives in the heads of a few experienced colleagues and is never written down. That second problem is getting sharper: nearly 40% of facility managers expect to retire within ten years, and the same pattern runs through trades, plants and engineering offices. FAM-WRK-01 · #18
A useful definition: onboarding ends when a person can do the core of their role to the organization's standard, without someone beside them, and the organization has evidence of it. Everything in this guide serves that finish line. It applies to new hires, internal transfers, people promoted into supervision, and the partner firms who join a project halfway through. #25 · #28 · #62
Three ideas run through the rest of the guide. First, design for the person's starting point: a graduate and a fifteen-year veteran need different things, and giving both the same programme makes it worse for both. Second, practise the real work, with the real templates, close to when it will be used. Third, verify before you trust, and measure what people can do rather than whether they liked the course. LSC-CLT-02 · LSC-TRF-02 · LSC-MSR-01
Most of what makes the first month go well is decided before the person arrives. Start with the role, not the course catalogue. Break the role into its recurring tasks and decisions, and for each one ask what the organization needs to observe when it's done well. A site engineer's tasks might include setting out, reviewing shop drawings, raising requests for information, checking a pour against the inspection plan and briefing a subcontractor. A plant technician's might include isolating equipment, changing a setpoint under change control and reading an alarm history. That list is the backbone of the plan. #13 · #34
Next, find out what the person already knows. A short diagnostic task, done in the first days or even before starting, tells you which parts to skip. The evidence here is unusually clear: heavy guidance that helps novices actually hinders experienced people, a finding first shown with electrical apprentices and PLC programmers. So build two tracks. Newcomers to a skill get demonstrations and worked examples. Experienced hires get problems and reference material, and a conversation about where their old habits will clash with your ways of working, such as drafting habits carried from 2D CAD into object-based modelling. LSC-CLT-02 · LSC-CLT-03 · #49
Then handle the practical set-up so day one isn't spent waiting:
Finally, write down what success looks like at day 30, 60 and 90 and record today's baseline. If you can't say what someone should be able to do in a month, you won't be able to tell whether they can. The 30-60-90 guide and builder in this series turns this into a template. #14
The first weeks should lead with the person and the real challenges of the role, not with a week of slides. Introduce the people they'll depend on and the decisions they'll face, then get them working on supervised versions of real tasks as early as possible. Onboarding programmes that open with the role's actual problems, rather than generic company content, give people a reason to learn the rest. GL-15 · #26
For tasks the person hasn't done before, use the sequence with the strongest evidence in technical training: show a complete worked example, then give partly completed tasks, then independent ones. In a review of 55 studies, learners who studied worked examples outperformed those left to solve problems unaided, with the typical example-learner doing better than about two in three of their peers. For a BIM coordinator that might mean watching a federation set up, then finishing a half-built clash test, then running one alone. For a site engineer it might mean reviewing a completed inspection plan, then completing one for a simple pour, then drafting one from scratch. LSC-CLT-01 · #59
Some things must be clear in week one regardless of role:
Keep the pace humane. Don't put a newcomer through everything in week one; spread it out, because spacing improves retention and because people need time to use what they've learned. And design the environment so that asking questions is normal. Psychological safety isn't a soft extra: it's the condition under which a new person tells you they don't understand the lifting plan before the lift. #3 · CS-06
The single most reliable finding in learning research is that retrieving something from memory strengthens it more than reviewing it again. Across 118 studies, practice testing beat restudying by a moderate margin, and by more when learners got feedback. For technical roles, retrieval doesn't mean a multiple-choice quiz. It means doing the task from memory: setting up the model, isolating the pump, completing the permit, then getting feedback on what was missed. LSC-PRC-01 · #19 · #60
Spread that practice out. Short refreshers at roughly one, three and eight weeks after first learning a skill, timed where possible to when the person actually uses it on a project, are far better than one long session. Don't pay extra for "optimal spacing algorithms": evenly spaced refreshers work as well as expanding schedules. LSC-PRC-02 · LSC-PRC-03
Expect the best methods to feel harder. Practising from memory, spacing sessions and mixing problem types all lower how well people perform during training while improving what they remember and transfer. Fluent, easy, massed practice feels better and works worse, which is why people left to pace themselves tend to reread rather than practise. Build the retrieval and spacing into the programme; don't leave them to choice. LSC-PRC-05 · LSC-PRC-06
Two cautions about technology. First, AI assistants can do the work instead of the learner. In a field experiment, students who practised with an unrestricted AI assistant performed better while using it and 17% worse once it was taken away; a guarded tutor that gave hints instead of answers avoided the harm. For new staff, use AI that prompts and checks, and periodically check that people can do the core task with it switched off. LSC-MOD-03 · PRM-LRN-01
Second, VR is a tool for specific jobs. Across studies with real-world outcomes, extended-reality training worked about as well as conventional training, not better, with a small edge for physical tasks. Use it where real practice is dangerous, expensive or impossible: confined spaces, fall protection, crane familiarization, a building that doesn't exist yet. Don't use it to teach software. LSC-MOD-01
Finally, protect the early-career experiences that used to come from routine work. As AI and automation absorb drafting, takeoffs and first-pass checks, juniors lose the repetitions that built judgment. Replace them deliberately with supervised cases, peer review and explanations of why. #54 · DM-WRK-04
Before someone works alone on anything consequential, confirm they can do it and interpret the result. That means watching them do a representative task to the organization's standard, not checking that a course was completed. For model-based work, it includes whether they understand what the model is telling them and who's accountable for it, not just whether they can operate the software. #28 · BM-ROL-04
The person who most shapes whether training turns into practice is the direct supervisor. Supervisor support and the climate of the team consistently predict whether skills transfer to the job, especially for judgment-heavy skills such as coordination and site leadership. Give supervisors a short guide: what the person has practised, what to give them to do next, and what to watch for. #53 · LSC-TRF-04
Let new people make mistakes where mistakes are safe. Error-management training, where learners are encouraged to make and learn from errors in a safe setting, improves transfer to new situations. A sandbox model, a test controller, a mock tender: give them somewhere to break things before anything real depends on them. LSC-TRF-05 · #44
Pair them with experience, in both directions. Mentoring is a retention lever as well as a learning one, and an experienced colleague recording how they actually do the work, while they're still there to do it, is the cheapest knowledge capture available. Newer staff often bring digital fluency the veterans lack; pair them so judgment and new methods both move. FAM-WRK-02 · #18
Most onboarding is evaluated with a satisfaction survey. Unfortunately, how much people liked a course barely relates to what they learned: ratings mostly reflect the instructor's style and how pleasant the experience was. High satisfaction can even accompany weaker designs, because the methods that work best feel harder. LSC-MSR-01 · #10
Measure what matters instead:
Set the measures before the first person starts, not after. If time to independent work currently averages five months for site engineers, write that down. A programme that brings it to three months has paid for itself several times over, and you can say so. A programme without a baseline can only report that people attended. For small firms that hire a few people a year, keep it simple: a one-page record per starter of when they first completed each critical task alone, and what went wrong on the way. #14 · LSC-MSR-03
Then close the loop. Each cohort's results should change the next cohort's programme: what to drop, what to move earlier, what new people kept getting wrong. Onboarding is a process you improve, not a document you publish. #64 · QMS-ARC-01
The checklist. Define ready. Diagnose the starting point and run two tracks. Use real templates and real tasks. Show, then fade. Practise from memory, spaced over weeks. Make stop authority and irreversible steps explicit. Verify before trusting. Give supervisors a role. Measure performance, later. Improve the next cohort.
Most organizations have a training catalogue. Few have a learning strategy: a deliberate link between what the business needs to be able to do, how people get there, and evidence that they did. This guide sets out the ideas, the practices worth keeping, and a roadmap.
A catalogue answers "what courses do we offer?" A strategy answers "what must our people be able to do in two years that they can't do today, and how will we know?" The second question starts with the business: a move to model-based delivery, a new plant line, a safety record that has stalled, a generation of experienced staff about to retire. #13 · #14
Three diagnostic layers keep the strategy honest. At the organization level: which outcomes matter, and which gaps in capability stand between you and them? At the task level: which recurring tasks and decisions make up each role, and which are changing? At the person level: who can already do what? Requests that arrive as "we need a Revit course" often turn out, after this diagnosis, to be a standards problem, a workflow problem or a staffing problem that training won't fix. #13 · #34
Learning is also socio-technical. A new tool, a new procedure or a new delivery method changes roles, decision rights, data, incentives and support all at once. Training that ignores those changes teaches people to work in a system that doesn't exist. Plan the operating change and the learning together, and expect a temporary dip in productivity while people adjust. #63 · #11
Finally, a strategy is a loop, not a launch. The same closed loop that quality systems use applies here: plan, deliver, check what changed, fix the cause of what didn't, and learn. Most learning programmes run the first two steps and skip the rest. #64 · QMS-ARC-01
Consider a typical request: "Our site teams need training on the new inspection app." Diagnosed at the organization level, the goal is fewer missed hold points and faster closeout. At the task level, only three steps actually change: recording an inspection, attaching evidence and releasing a hold point. The person level shows that most supervisors already use the app's photo features and few know the release workflow. The result is a 40-minute practice session on real inspections, timed for the week before go-live, plus a one-page job aid. It isn't a day-long course, and it costs a fraction of what was asked for. #13 · QMS-ITP-02 · #22
The strongest evidence for training adults in technical work comes from cognitive and instructional psychology, not from neuroscience. Five findings are robust enough to design around:
Two cautions keep these honest. Most of the evidence comes from students learning words or mathematics, so effects for complex field and software skills are probably smaller. And the methods that work best often feel harder, so satisfaction ratings will reward the wrong designs. LSC-EVD-02 · LSC-PRC-05
Several popular claims should come out of your materials and your vendors' pitches:
What does a design built on the robust findings look like? Take a two-week programme on model coordination for project engineers. Day one: study two complete clash-review records from past projects, then a third with the decisions blanked out. Day two: run a live clash review on a real model, from memory, with a coach. Then, instead of a second day in the classroom, three 20-minute refreshers at the end of weeks one, three and eight, each a new review on the engineer's own project. Experienced coordinators take a short diagnostic review on day one and skip the worked examples if they pass. None of this needs new technology; it needs a different schedule. LSC-CLT-01 · LSC-PRC-01 · LSC-PRC-02 · LSC-CLT-03
The practical consequence: build your method on the robust findings, label weaker claims for what they are, and ask every vendor for learning results on real tasks rather than vocabulary. LSC-EVD-01 · CM-EVD-01
The skills that matter most rarely sit inside one discipline. A BIM coordinator needs information management, contracts and coordination. A plant technician now needs controls, data and security. A project manager needs estimating, procurement and people. A learning strategy that mirrors the org chart will miss the seams where work actually fails. #20 · #47
Four capability shifts show up across the sectors in the Database:
Map these into a competency framework that goes beyond technical operation: judgment, communication, explaining decisions, ethical use of AI and data. Tool proficiency alone doesn't show readiness for professional work. #36 · #42
A practical way to find the seams is to trace one recurring failure back through the teams it touched. A late steel change, for example, may involve a design team that issued without coordination, a procurement team that didn't flag the lead time, a site team that found out at delivery, and a commercial team that missed the notice deadline. Each team's own training covered its own step. Nobody was trained on the handoffs. Cross-disciplinary learning means building scenarios, rotations and joint reviews around those handoffs, so a site engineer has sat through a procurement review and a buyer has walked a pour. #20 · VCM-ADM-01 · CM-RSK-02
Format follows the task. A few rules from the evidence:
Whatever the format, build it on your own standards and templates, and time it close to when people will use the skill. Software training delivered months before a rollout largely evaporates. #16 · LSC-TRF-03
Most strategies end up with a blend: a short, shared foundation for everyone, simulations for the judgment-heavy roles, on-the-job practice with coaching for the procedural ones, and a community that keeps it current. The mix is right when each piece can name the task it serves. #61 · #53 · #33
A roadmap turns the strategy into sequenced commitments. This one is deliberately simple; adapt the timing to your size.
Fund it properly. Communications, coaching, governance and reinforcement cost money, and programmes budgeted only for content tend to stall at completion. Visible leadership commitment, including protected learning time, is a recurring difference between programmes that change behaviour and those that don't. #9 · #7 · QMS-LSS-02
Two warnings for the roadmap. First, resist scaling before the pilot has a result; enthusiasm from the pilot team is not a result. Second, expect the roadmap to change. A new owner requirement, a plant expansion or an AI tool that changes a workflow can reorder priorities overnight. Review the roadmap quarterly against the business outcomes you set in the first three months, and drop work that no longer serves them, however much has been invested. #1 · #57 · #14
A learning strategy is credible only if it reports results the business recognizes. Use the four levels as a taxonomy, but stop at reaction only if you have to: performance on job-like tasks after 30 to 90 days and observed use on the job are what matter. #10 · LSC-MSR-03 · LSC-MSR-04
Prefer objective records to self-report: model audits, inspection results, rework, nonconformances, system logs, supervisor observation. Use comparison groups or staggered rollouts where you can, because pre-post designs can't separate training from everything else that changed. LSC-MSR-05 · QMS-ARC-04
Be clear about learning data. Tell people what is collected, who sees it and whether it affects assignments, and keep AI-generated recommendations explainable and contestable. Trust in the system is a precondition for honest participation. #48 · #50
Governance doesn't need a large structure. A small learning council works well: an operations leader who owns the business outcomes, a technical lead from each major discipline, the person who runs learning, and someone who speaks for the frontline. It meets quarterly, reviews results against the baselines, approves what scales and what stops, and holds owners to their content reviews. Between meetings, each programme has a named owner who answers for its results, not just its completion rate. #7 · #57 · LSC-MSR-01
Finally, publish what you find, internally at least. Rigorous evidence on training in construction, engineering and manufacturing is scarce. An organization that measures its own programmes properly knows more than its competitors, and can prove it. LSC-MSR-06
The short version. Start from business outcomes. Diagnose at three levels. Build on retrieval, spacing, worked examples, two tracks and real templates. Retire the myths. Develop capability at the seams between disciplines. Pick formats by task. Pilot, measure, then scale. Fund the follow-up. Report performance, not satisfaction.
Project management in architecture, engineering and construction isn't general project management with hard hats. It's a chain of commitments made under uncertainty, across firms, under contracts and payment law, with information that has to outlive the project. This guide follows that chain in the order a project meets it.
Every project begins with a number, and the number is usually optimistic. In a database of more than 16,000 projects, under half came in on budget, 8.5% came in on budget and on time, and half of one percent also delivered the benefits promised. Those figures vary by sector and country; some transport programmes overrun far less. The point isn't that projects are doomed. It's that the first estimate is a forecast, and forecasts should start from what usually happens. CM-RSK-01
The discipline that corrects for this is reference-class forecasting: find completed projects like this one, see how they actually finished, and adjust from there. If your last six healthcare jobs finished 11 to 24% over their bid estimates, a new one priced as if it won't is a bet, not an estimate. CM-RSK-02 · FIN-FPA-01
Three practices make the forecast honest:
Finally, check who benefits from the number. Optimism needs a counterweight, and estimators paid a bonus on winning work are, without any dishonesty, paid to be hopeful. Know your bonding and cash capacity before you bid, and rehearse what happens to the company if this job goes like the last bad one. FIN-CON-01 · FIN-TAL-06 · FIN-CON-05
Delivery methods are often chosen by habit or fashion. The evidence says to choose by the project. Design-build reliably saves schedule; its cost advantage depends on project type, and on uncertain ground traditional design-bid-build can give better final cost predictability, because lump-sum design-build bids carry large risk premiums. Integrated project delivery shows promising results in small samples and needs mature organizations and shared risk pools. CM-DEL-01 · VCM-DEL-01 · VCM-DEL-02
Where scope is genuinely uncertain, progressive design-build lets the owner and design-builder develop the design, budget and schedule together before the price is fixed, with an off-ramp for both sides. Standard forms now exist for it: AIA A141PDB-2024 in the US and CCDC 32 (2026) in Canada. CM-DEL-06 · VCM-FRM-03
Whichever model you choose:
At mobilization, get every party to state what they'll deliver, who decides what, and what success looks like. Differences resolved before production cost a fraction of differences discovered during it. #62
A short example shows why the choice matters. A municipal owner wants a new transit maintenance facility on a brownfield site with unknown contamination. A lump-sum design-build bid at concept stage would price the ground risk heavily, or ignore it and fight about it later. Progressive design-build lets the team investigate the site, develop the design to a point where the ground risk is understood, and then agree a price, with the owner keeping the right to walk away and tender the remaining work if the price doesn't hold up. The same owner building a standard bus shelter programme should simply tender it. Same owner, same year, two different answers, because the projects differ. CM-DEL-01 · CM-DEL-06 · CM-RSK-04
A building or bridge outlives its project by decades, and so does its information, or doesn't. On ISO 19650 projects, BIM management means running the information management process, not running the modelling software. Contract to the current 2018 editions of Parts 1 and 2; the 2026 revisions are drafts. BM-STD-01
The process starts with requirements and flows downward. The owner's organizational and asset needs define project requirements, which become the exchange requirements issued at tender and passed down to every appointed party. If the facilities team isn't asked what it needs to operate the building, the model will contain what designers find interesting instead. BM-REQ-01 · FAM-SYS-01
Four practices do most of the work:
Treat the BIM execution plan submitted at tender as evidence of capability, and keep the confirmed plan as the working governance document. The most common failure is never touching it again after signing. BM-REQ-02 · BM-REQ-03
The most useful single step is to put the operator in the room early. Ask the facilities manager or plant engineer which assets they maintain, what data their maintenance system needs for each, and what they had to collect by hand on the last project. Their answers usually fit on two pages and save months of rework at handover. The project manager's job is to make sure those two pages turn into contract requirements, and that someone checks the deliveries against them at each stage, not only at the end. FAM-SYS-01 · BM-QA-04 · FAM-DAT-04
Production control is where schedules are kept or lost. The Last Planner System replaces "push" schedules with commitments made by the people doing the work: phase pull-planning, a six-week lookahead with a constraint log, and weekly commitments only on tasks whose constraints are cleared. CM-LEAN-01
Track Percent Plan Complete, the share of weekly commitments kept. The industry unknowingly plans at about 50%; mature users aim for 70 to 80%, and projects that finished on schedule showed higher, steadier PPC from early on. Feed the reasons commitments were missed into the risk register: materials, design, prerequisite work and weather are risk signals, not excuses. CM-LEAN-02 · CM-LEAN-03
Coordination needs decision-makers in the room. Weekly or fortnightly federated reviews with people who can approve changes resolve in a meeting what otherwise waits a week. Report clashes as clusters by system and level, give every issue an owner and a date, and record which RFIs and change orders each resolved clash prevented. Where a contractor traced clashes to actual RFIs and change orders, savings were modest, about 0.1% of project cost, but still a 127% return. BM-COO-04 · BM-COO-03 · BM-COO-05 · BM-COO-06
Quality needs controls where mistakes can't be undone. Write an inspection and test plan for each work package, put hold points before grout, concrete or cladding hides the evidence, and define hold, witness and review points in the contract. When something fails, fix the immediate problem and, separately, the cause; and close the corrective action only when you've shown it worked, not when it was implemented. QMS-ITP-01 · QMS-ITP-02 · QMS-ITP-03 · QMS-CAP-01 · QMS-CAP-03
Safety rides on the same signals. Schedule pressure drives risk-taking, so read missed commitments and acceleration requests as safety information. Track leading indicators, such as pre-task plan quality, near-miss reports and how quickly corrective actions close, rather than only recordable injuries. CM-SAF-06 · CM-SAF-02
Cash is a project's bloodstream, and payment law sets its pulse. Under Ontario's Construction Act, for example, an owner must pay a proper invoice within 28 days, must give a notice of non-payment with reasons within 14 days or owe the invoice in full, and contractors must pay subcontractors within 7 days of being paid; adjudication decisions bind in the interim. Other jurisdictions have their own clocks, and the UK has a bill to phase out cash retentions. Know yours. VCM-PAY-01 · FIN-CON-03 · VCM-PAY-02
Most disputes are controllable. In consultancy data, the leading causes are errors and omissions in contract documents and missed contractual notices, both of which come down to discipline. So:
Measure contract value loss from your own ledger (unrecovered changes, late-payment penalties, dispute costs) rather than quoting an industry "leakage" percentage that its own publisher says can't be validated. VCM-ADM-03 · VCM-EVD-01
A notice register doesn't need software. A shared spreadsheet works if it has five columns: the clause, the trigger event, the deadline in days, the person responsible and the date the clock started. What makes it work is the habit: a ten-minute review at every weekly project meeting, and a rule that anyone who sees a trigger event, such as a late drawing, an instruction on site or an invoice not paid, adds a line the same day. Projects that lose money on notices rarely lack the right; they lack the date. VCM-ADM-01 · VCM-ADM-02 · VCM-PAY-01
The decisions that define a project are usually made late in the day, with incomplete information and someone senior wanting a particular answer. A few habits change how those decisions go.
Picture a familiar Friday. The concrete supplier has warned that Monday's pour may be short of trucks. The superintendent wants to go ahead; the project manager has a client meeting promising the slab; the quality manager hasn't seen the final rebar inspection. A good decision process takes fifteen minutes: each person states the facts they hold before anyone recommends; someone is asked to make the case for postponing; the hold point is checked, not assumed; and the decision and its reasons go in the log. Whichever way it goes, the team can explain it on Monday. CS-08 · PM-22 · QMS-ITP-02
None of these slows a good team down. They're the difference between a decision you can defend afterwards and one you have to explain.
The chain. Forecast from base rates. Classify the estimate and know your capacity. Choose delivery by the project and administer the form you chose. Plan the information from the operator's needs. Control production with commitments and PPC. Put quality controls where mistakes can't be undone. Run a notice register. Pool facts, invite dissent, then decide.
A 30-60-90 plan stages the first three months in a new role, on a new site or with a new system: learn, then contribute, then own. Done well, it makes progress visible to the person and their supervisor. Done badly, it's a to-do list nobody reads after week two. This guide explains the difference and ends with a builder.
A 30-60-90 plan answers one question for each stage: what should this person be able to do by then, and how will we both know? It's not a list of meetings to attend or modules to finish. It describes growing capability: in the first month, learning how the work is done here; in the second, contributing to real work under supervision; in the third, owning a defined part of it to the organization's standard. #25 · #28
The format suits more situations than new hires:
Two principles keep it useful. First, write it with the person, not for them, starting from what they already know. A diagnostic task in week one tells you which parts to skip, and experienced people given a novice's plan disengage, with good reason. Second, attach a checkpoint to each stage: a short conversation at days 30, 60 and 90, based on observed work, where both sides agree what changes next. LSC-CLT-02 · #49 · #14
The wording of each item matters more than its length. Compare "Attend BIM orientation" with "Set up a federated model for Level 3 and run the agreed clash tests, reviewed by the BIM manager." The first records attendance. The second describes something the person will be able to do, on real material, with a named reviewer, and both of you will know whether it happened. Write every item that way: a verb that describes doing, a real piece of work and a way to check it. If an item can't be written like that, ask whether it belongs in the plan at all. #28 · LSC-MSR-03
Keep the plan short. Five or six items per stage is plenty. A plan with forty items signals that everything matters equally, which means nothing does, and it won't be opened after the first fortnight.
The first month is about the person, the people and the real challenges of the role. Start with introductions to the people they'll depend on, the decisions they'll face and the risks that matter here. Then move quickly to supervised versions of real tasks. GL-15
For a new site supervisor, the first week might look like this. Day one: site induction, then a walk of the work area with the outgoing or senior supervisor, naming every hold point and stop condition on the way. Days two and three: shadow the morning pre-task briefings and read three completed pre-task plans from the past month. Day four: draft the next day's pre-task plan, with the senior supervisor correcting it before it's used. Day five: meet the inspector and the main trade foremen. By the end of the week the person has done real work, under supervision, and knows who to call. Most onboarding spends that week on slide decks. SOP-USE-03 · LSC-CLT-01 · #62
Day-30 checkpoint: what has the person done, not just seen? What still confuses them? Which parts of the plan can be dropped because they already knew them?
The second month moves from watching to doing. Give the person a bounded, real deliverable with a supervisor reviewing it: a section of a model, a pour package, a set of preventive-maintenance routines, one subcontractor's submittals. Pilot on something real but contained before widening their responsibilities. #1 · #2
The size of the month-two deliverable is a judgment call. Too small, and the person learns nothing about the role's real pressures; too large, and a mistake lands on the project. A good rule is "real, reviewed and recoverable": the work is used on the project, someone experienced reviews it before it's relied on, and any error can be caught and fixed before it costs much. A BIM coordinator might run federation and clash reports for one zone while the BIM manager runs the coordination meeting. A maintenance technician might carry out preventive routines on non-critical equipment before moving to the critical list. #1 · BM-COO-02 · FAM-MNT-01
Day-60 checkpoint: review the deliverable together. What did the person do independently? What still needed correction? What will they own in month three?
By the third month the person should own something end to end, with a supervisor checking outcomes rather than steps. #28
Day-90 checkpoint: is the person working independently in the agreed scope? Record it, and record what the next cohort's plan should change.
Checkpoints work best as short, structured conversations around real work. Bring the deliverable, not a feedback form. The supervisor asks three questions: what went well, what was hard, what should change in the plan. The person answers a fourth: what do you need from us? Write down the agreed changes in the plan itself, so it stays a working document. Twenty minutes is usually enough. If a checkpoint keeps getting postponed, that tells you something about how much the plan is valued, and that should be fixed before the next starter arrives. #53 · #49
Common mistakes are worth naming. Plans that list activities rather than outcomes. Identical plans for novices and veterans. All the learning front-loaded into week one. Checkpoints that ask "how's it going?" instead of reviewing work. And measuring success by satisfaction rather than by what the person can now do. LSC-MSR-01 · LSC-PRC-06
The builder below starts from one of six templates. Each follows the same learn, contribute, own pattern with tasks drawn from the role. Pick the nearest one and edit it.
For experienced hires and internal moves, the builder swaps the worked-example steps for a diagnostic task and problem-first practice, because guidance that helps novices gets in the way of experts. LSC-CLT-02
Three adaptations come up often:
Choose a template and a start date. Tick items as they're done; your ticks are kept on this device only, and nothing is sent anywhere. Print it or copy it into your own documents.
Treat the output as a first draft. Sit down with the person in their first week, read it together, cross out what they already know, add what's specific to your site, plant or project, and agree the checkpoint dates in both calendars. The refresher dates are suggestions; move them to when the skill will actually be used. And at day 90, keep the marked-up copy: it's the best evidence you have for improving the next person's plan. LSC-PRC-02 · #64
Risk is mostly decided before anything goes wrong: in the estimate, the contract, the design, the controls, and the way back you did or didn't build. This guide draws on construction, plants, software, cyber and contracts, because the habits that avoid risk are remarkably similar across all of them.
The cheapest moment to avoid a risk is before you've committed to it, and the most common error at that moment is optimism. Planners take the "inside view", building up from the details of this project, and the inside view almost always comes in low. The correction is the outside view: how did similar projects, plants or rollouts actually go? Start there, then adjust. CM-RSK-02 · CM-RSK-01
Three habits make early risk visible:
Be wary of the famous numbers used to justify decisions. "1-10-100", "70% of transformations fail", "9.2% value leakage", "rework is 30% of construction": each is either folklore or a figure stripped of its context. Build the case from your own data and named sources. QMS-COQ-01 · VCM-EVD-01 · QMS-RWK-03 · CM-EVD-01
Consider a manufacturer deciding whether to add a second production line. The inside view says eighteen months and a known budget, built up from equipment quotes. The outside view asks how the last three line installations in the company, and comparable ones at peer plants, actually went: how long commissioning took, how long it took to reach rated output, what broke in the first year. If the answer is "six months longer than planned, and output at 70% for the first quarter", that belongs in the business case now, along with the training and maintenance staffing that the quotes leave out. CM-RSK-02 · DM-WRK-04
Not every step deserves the same scrutiny. The principle that runs through quality, safety, plants and software is the same: tier controls by consequence, and concentrate them where a mistake can't be reversed. QMS-ARC-03
The corollary is just as important: don't spend control effort on what's cheap to undo. Teams that put every step through the same heavy process train people to treat all controls as paperwork, including the ones that matter.
Finding the irreversible steps is a short exercise, best done with the people who do the work. Walk through the process, whether a pour, a cutover, a shutdown or a contract award, and at each step ask: if this is wrong, can we undo it, and at what cost? Mark the steps where the answer is "no" or "only at great cost". Those are where the hold points, second checks, simulations and sign-offs belong. Everything else gets the lightest control that works. Most teams find that a handful of steps carry most of the risk, and that some of their heaviest controls sit on steps that are easy to reverse. QMS-ARC-03 · QMS-ITP-01
Many risks can't be prevented, only made recoverable. The question to ask of every plan is: if this goes wrong, how do we undo it, and have we tried?
A way back you haven't rehearsed is a hope. Exercises, tabletops and restore tests turn it into a capability. CYBER-13 · GL-16
Rehearsals don't need to be elaborate. A ninety-minute tabletop once a quarter, built around a realistic scenario such as the scheduling system down on pour day, a key supplier insolvent or a control system change gone wrong, will expose the gaps: the phone numbers nobody has, the backup nobody has restored, the decision nobody knows they own. Write down what you found, fix the three most important gaps, and run a different scenario next quarter. The goal isn't a perfect performance; it's finding the problems while they're cheap. CYBER-13 · CYBER-01 · #61
Contracts and supply chains decide who carries which risk, whether anyone meant them to or not. Make the allocation deliberate:
Choose security instruments knowingly, too. Surety bonds and subcontractor default insurance move different risks to different parties, and no independent data compare their default rates. VCM-PAY-03
The cost of leaving this to chance shows up when a key subcontractor gets into trouble. If the mechanical subcontractor on a hospital job stops paying its suppliers, the general contractor who tiered its vendors already knows it's a critical trade, has been watching its payment behaviour, holds a bond or default insurance chosen for this risk, and has a notice register that shows exactly which deadlines are running. The one who didn't finds out from a supplier's lien, scrambles for a replacement at a premium, and discovers the contract never said who carries the delay. VCM-SUP-01 · VCM-PAY-01 · VCM-ADM-01
Lagging measures such as injuries, overruns, outages and claims tell you about risk after it's arrived. Leading signals give you time to act:
A signal is only useful if someone acts on it. Keep a risk register small enough to read, delete risks nobody would act on, and tie numbers to actions agreed in advance. FIN-RSK-03 · FIN-RSK-04
Signals also travel across disciplines, which is where they're most often lost. A falling Percent Plan Complete is a schedule signal and a safety signal. A run of rushed corrective actions is a quality signal and a sign of a team under pressure. A vendor missing deliveries is a supply signal and possibly a solvency signal. A short weekly review that puts the site, quality, commercial and safety signals on one page, with the people who own each in the room, catches combinations that each discipline's report alone would miss. CM-LEAN-03 · CM-SAF-06 · #20
Even well-run organizations have bad days. What separates a bad day from a disaster is mostly decided before it starts:
Imagine a plant at 2 AM: a packaging line stops, the control screens show alarms nobody recognizes, and the night supervisor can't reach the controls engineer. A prepared site has a one-page plan on the wall that says who decides whether to shut the line down, how to run in a safe manual mode, and who to call next. The supervisor makes the call, logs the time and reasons, isolates the line, and posts an update to the morning shift. Nobody is guessing, and the investigation the next day starts from a clear record rather than recollections. CYBER-01 · CYBER-11 · CYBER-18
Afterwards, learn properly. Review the conditions that combined, not the person who happened to be nearest; root-cause tools generate hypotheses, so require evidence for each link. Then close the loop: give actions owners and dates, and check whether the problem came back. The most common failure in quality, safety and software alike is an action closed on paper that never prevented anything. SWD-INC-05 · SWD-INC-06 · QMS-RCA-02 · QMS-CAP-03 · SWD-INC-07
One last habit ties the guide together: share what you learned beyond the team that had the incident. The site that nearly lost a crane lift, the plant whose backup wouldn't restore and the project that missed a notice deadline each learned something other teams will need. A short, blame-free summary of what combined to cause the event, and what changed as a result, circulated across disciplines, turns one team's bad day into everyone's warning. Organizations that do this consistently find that their near-miss reports go up and their serious events go down, because people see that reporting leads to fixes rather than to blame. CM-SAF-02 · #33 · #64
The habits. Start from the outside view. Run a pre-mortem. Concentrate controls where mistakes can't be undone. Build and rehearse a way back. Allocate risk in writing and keep a notice register. Watch leading signals and act on them. Pre-assign decisions, mitigate first, communicate early, and learn without blame.
Most of what goes wrong after a go-live was decided in the weeks before it: a readiness check that measured opinions instead of behaviour, a contingency plan written for an outage nobody had, and a hypercare period that ended on a date instead of on the data. This brief covers the switch-over and the months after it, for software, plant systems and asset records alike.
A readiness review built from status reports and surveys tells you what people believe about the change. It doesn't tell you whether the change works. Canada's Phoenix pay system is the case to remember: the planned pilot was cancelled without an impact assessment, the system was never tested as a whole, and a fifth of the functions reviewed failed testing and weren't retested. A year after go-live, about half of employees had errors in a single pay run. DGN-RDY-03
Make a behavioural test a gate criterion: a pilot, a dress rehearsal or a parallel run, sized to the risk. If someone cancels it, require a written impact assessment and a fresh decision from the accountable executive. Read readiness as two things, commitment to the change and confidence in doing it together, and report how it varies across sites and roles, not just the average; the site that scores lowest is where the first week goes wrong. DGN-RDY-01 · DGN-RDY-02 · SWD-REL-01
If scope is cut late, the benefits case and the readiness diagnosis were made for a different project. Re-run them. DGN-RDY-04
For physical assets the same rule applies to the records: commission the information with the asset. A pump station handed over with a working pump and an asset register that doesn't match it hasn't gone live; it has moved the problem to the first night-shift call-out. UTL-REQ-04
Phoenix had a contingency plan, finalised less than two weeks before launch, and it mostly covered what to do if the system failed to operate. What actually happened was worse and slower: a system that ran, produced wrong results, and couldn't keep up with the volume. GOL-HYP-04
Write the contingency for process and volume failures: the manual workaround, the paper fallback, the extra staff, and the trigger that switches them on. Rehearse the trigger, not just the plan. Before automating, simplify and standardise the rules; automating a messy process makes the mess faster. GOL-HYP-04 · GOL-HYP-07 · CYBER-11
| Risk tier | Examples | Before go-live | Hypercare |
|---|---|---|---|
| Low | Dashboards, templates | Smoke test and a rollback plan | About two weeks of watching |
| Moderate | Common data environment, RFI and submittal tools, field apps | A pilot crew | Four to six weeks |
| High | ERP, payroll, MES | A full pilot, tested and retested; a volume contingency | Clean reconciliations for two business cycles |
| Safety-critical | SCADA, new lines, fall protection, lockout | Hazard analysis and independent sign-off | Observed procedure fidelity; exit on criteria only |
A design synthesis, not a validated method: use it as a starting point and adjust from your own stabilisation data. In every tier, the calendar triggers a review; the data decides the exit. GOL-HYP-08 · GOL-HYP-01
The most expensive habit in hypercare is filing everything under "user training". Phoenix's department initially believed the problems came from other departments not knowing how to use the system. A support desk that sends every problem to training hides the defects, and the people who reported them stop reporting. GOL-HYP-02 · DGN-FRM-03
Report the mix weekly. A hypercare where "training" keeps rising while "defect" stays flat is usually misclassifying. GOL-HYP-02 · GOL-KCS-01
Give bad news a route of its own. Phoenix's auditor linked the failure to public servants afraid to tell their political masters about problems. Set up a channel for frontline staff and the support desk that bypasses the project's own sponsors, and log what's reported and what's decided. GOL-HYP-06
No source supports a standard hypercare length, including the popular 30 days. Exit when the data says so: ticket arrivals back to the forecast baseline, no open severity-1 or severity-2 defects for a set period, backlog age under target, and workflow errors within tolerance. If a contract fixes the hypercare term, honour it, but still report stabilisation. GOL-HYP-01
In a phased rollout, let each wave's stabilisation data gate the next wave. A second site that goes live while the first is still unstable splits the support team and doubles the backlog. GOL-HYP-05
Expect a productivity dip, but don't plan around a folklore number. The widely quoted "plan for a 30% drop" is trade-magazine opinion; real studies range from no long-term loss to a loss that never recovered. Baseline productivity before go-live, then report the trend by role over twelve months rather than one snapshot. GOL-HYP-03 · DGN-BSL-01
"Hypercare closes Friday; the project team rolls off Monday." The backlog moves to a general help desk with no context, and severity-2 defects become permanent workarounds.
"Hypercare closes when tickets are back to forecast, there have been no open severity-1 or -2 defects for ten working days, and the backlog is under five days old." The date is a forecast, not a promise.
Champions help, modestly. A Cochrane review of local opinion leaders found they probably improve compliance with good practice by a median of about 11 percentage points, with a wide range, and mostly among physicians. There's no evidence for any champion ratio, including the popular one per twenty users. Where the problem is a system defect, a champion can't fix it. GOL-CHP-01 · LSC-MNT-05
Skills fade, buildings drift and workarounds come back. Plan for relapse from the start. GOL-ADP-03
The habits. Make a behavioural test a gate. Plan for the system that runs wrong, not only the one that stops. Sort every ticket into defect, data, process, training or access. Give bad news its own channel. End hypercare on stabilisation data. Give champions a written job and judge them on errors and rework. Measure adoption in the work, and plan refreshers before the skills fade.
An internal AI assistant is only as good as the knowledge it reads, the permissions it respects and the tests you ran before anyone trusted it. This brief is a pre-launch checklist for firms putting an assistant over their standards, procedures, project files or asset records, written for engineering, construction and manufacturing teams.
Start with a task, not a tool. Decide whether AI should do the task at all, then tier oversight by risk: drafting a meeting summary and answering "what's the cover to rebar on this element?" are different jobs and deserve different controls. PRM-BIZ-05 · PRM-GOV-02
An assistant over an ungoverned shared drive will retrieve the superseded standard as confidently as the current one. Retrieving a withdrawn standard faster is a risk amplifier, not a service. KIN-GOV-04
Grounding an assistant in your documents reduces invented answers; it doesn't remove them. In the first preregistered evaluation, commercial legal research tools grounded in curated sources and marketed as hallucination-free were wrong 17 to 33% of the time, and incomplete far more often. KIN-TWN-01 · KIN-GOV-02
Re-run the set periodically, and after any content or model change. Retrieval systems fail at the joints: missing content, the right document ranked too low, an answer lost while combining sources. KIN-TWN-04 · PRM-EVD-04 · AIG-09
Configure the assistant to give source, version and owner on every answer, and to say "not found" rather than improvise. Then make people open the citations: unread means uncited. KIN-TWN-01 · AIG-09
An index that ignores access controls turns every user into an administrator. Enforce source permissions at retrieval time, and before release, and after any index or permission change, red-team for answers that carry one project's or client's information into another's. No peer-reviewed leakage rates exist yet; this is implementation judgment, and it's cheap to test. KIN-TWN-05
Client confidentiality. Tag client-restricted content at source, honour contractual reuse limits, sanitize lessons before they're published across the firm, and check client contracts for AI clauses before the assistant reads a single file. Disclose AI use to clients in the contract, and ask your broker in writing about AI exclusions. KIN-TWN-06 · PRM-GOV-04 · AIG-10 · PRM-GOV-05
Search logs and expertise maps are people data. An assistant's logs show who asked what, and how often. In Germany, systems capable of monitoring behaviour or performance need works-council agreement; in France the works council must be consulted on new technologies. Under the EU AI Act, AI that allocates tasks or evaluates people's performance is high-risk, and employers must tell workers first. Quebec's Law 25 requires notice before profiling, including of work performance. Aggregate, minimize, set retention, and give people a way to contest. Canadian federal, UK and US law weren't part of this review: check them for your sites. KIN-GOV-06 · KIN-SIG-01
Keep prompts, sources and outputs as records where the work needs them, and read search and question signals as hypotheses about the organization, not verdicts about individuals. PRM-GOV-03 · KIN-SIG-01
The engineer stays responsible; AI never seals. That's easy to write and harder to keep when the assistant is right most of the time. PRM-ENG-01 · DQ-AI-03
The checklist. A named owner and a pause button. Only governed sources in the index, with authority shown. A gold set with unanswerable questions, re-run after every change. Source, version and owner on every answer, and "not found" when it doesn't know. Permissions enforced at retrieval and tested for leakage. Client contracts checked. Logs handled as people data. Skills checked with the AI off.
When you're the engineer, the project manager and the office, your methods live in your head and your inbox, and every contractor learns them from scratch. This brief shows how to write them down once, in a size that fits one person: a few procedures, a contractor system, quality checks at the points that matter, and a learning plan. The running example is an independent mechanical engineer who designs systems and works with contractors to build them.
Don't try to document everything. List the work you repeat, then mark the jobs where a mistake is expensive, unsafe or can't be undone. Standardize those first. SOP-SYS-06
A solo firm needs the same structure as a large one, at a much smaller scale. Keep a simple hierarchy: a short policy, a handful of procedures, and the forms and records they produce. A completed checklist is a record, not a procedure. SOP-SYS-02 · SOP-SYS-03
Test each procedure on real work before you rely on it; the first draft is always missing a step. SOP-DES-06
Your contractors are your workforce, so treat them as part of your system rather than outside it. Tier them by what happens if they fail, and spend your checking time accordingly. VCM-SUP-01
You can't inspect everything, so decide in advance what you will see. Write a simple inspection and test plan for each work package, and put hold points at the stages that can't be undone, before insulation, ceilings or concrete hide the work. QMS-ITP-01 · QMS-ITP-02
Licensure hours are a floor, not a measure of competence. Know your regulator's model and check it with them, then spend your continuing-education time on what actually goes wrong in your kind of work. LSC-CPD-01 · LSC-CPD-02 · LSC-CPD-05
A system for one person survives only if it costs little to maintain. Book the time as a non-billable line, like insurance, and let real events drive the changes. LSC-ORG-02
The checklist. Three procedures for your highest-consequence work, on one template, tested on a real job. One current version wherever contractors work. A register of codes and editions. An approved contractor list with tiers, flow-downs and a notice-deadline register. Hold points at the irreversible stages, written into contracts. A peer reviewer for high-consequence work. A learning plan aimed at real failures. Time booked to keep it all current.
General guidance for professional practices, not legal or engineering advice. Contract, payment and licensing rules depend on your jurisdiction and your regulator.
Every field has statistics that everyone has heard and nobody has checked. Building the Database meant tracing hundreds of them to their sources. These 40 didn't survive: some had no source at all, some were one study stretched far past its sample, and some were vendor claims that hardened into fact. Here's what each one actually rests on, and what to say instead.
A made-up number in a slide deck isn't harmless. It sets budgets ("plan for a 30% drop"), schedules ("hypercare lasts 30 days"), design rules ("learning styles") and expectations ("AI makes developers 55% faster"). When reality doesn't match, the project looks like it failed when it was the number that was wrong. CM-EVD-01
The rule the Database follows is simple: don't repeat a statistic you can't source, and when you can source one, keep it attached to its population, method and year. Each claim below appears in the Database's retired-claims lists, with the entry that replaces it. CM-EVD-01 · FAM-EVD-01 · PRM-RTS-02
| Stop quoting | What it actually rests on |
|---|---|
| 1. "Only 10% of training transfers to the job." | A 1982 rhetorical opener, re-cited as data. |
| 2. Learning styles (visual, auditory, kinesthetic) as a design rule. | Matching teaching to a preferred style doesn't improve learning. |
| 3. "Attention spans are 8 seconds." | No meta-analytic support, and no basis for fixed "brain-based" module lengths. |
| 4. "People remember 20% of what they read and 80% of what they see." | No research basis for these figures. |
| 5. "70-20-10" as a budget rule. | Executives recalling what developed them. A reminder, not a measurement. |
| 6. "It takes 21 days to form a habit." | No supporting evidence was found. |
| 7. "The forgetting curve means 90% is lost in a week." | Ebbinghaus studied one person learning nonsense syllables. |
| 8. "VR training is 4x faster" or "75% retention". | Uncontrolled vendor claims; overall, VR matches conventional training. |
| 9. "Microlearning improves retention by X%." | No direct comparisons support it. |
| 10. "Gamification increases retention by up to 40%." | A vendor blog with no study cited. |
| 11. "The 10,000-hour rule." | An average from one music study, turned into a threshold. |
| 12. "Our training returned 543%." | Under the ROI method's own conservative rules, the same example returns about 71% in the first year. |
Say instead: "Practice from memory, spaced over time, with feedback, has strong evidence. We'll check transfer on the job with supervisors and records, not a satisfaction survey." LSC-MYTH-01 · LSC-MYTH-03 · LSC-EVD-02 · GOL-TRF-03
| Stop quoting | What it actually rests on |
|---|---|
| 13. "70% of change efforts fail." | Traced to five sources in 2011; none held valid, reliable evidence. Success rates depend on definitions. |
| 14. "Hypercare lasts 30 days." | No source supports a fixed length. |
| 15. "One champion per 20 users." | No evidence supports any champion ratio. |
| 16. "Plan for a 30% productivity drop at go-live." | Trade-magazine opinion; studies range from no long-term loss to a loss that never recovered. |
| 17. "Overlearning prevents skill decay." | Contested. Schedule refreshers anyway. |
| 18. "94% of problems are the system, 6% the people." | Deming's personal estimate, not a measurement. Keep the principle, drop the number. |
| 19. "Decide when you have 70% of the information." | No controlled evidence, and only ever meant for reversible choices. |
| 20. "Pre-mortems make predictions 30% more accurate." | The study found about 30% more reasons for failure; accuracy wasn't measured. |
| 21. "Half of all decisions fail." | The source could not be found. |
| 22. "Unsafe acts cause 90% of injuries." | A framing Texas City used before 2005; investigators found systemic failures. |
Say instead: "We'll end hypercare when tickets are back to forecast and there are no open severity-1 defects. We'll measure our own productivity dip by role, against a baseline." GOL-HYP-01 · GOL-HYP-03 · QMS-LSS-04 · DGN-FRM-03
| Stop quoting | What it actually rests on |
|---|---|
| 23. "98% of megaprojects overrun by 80% and 20 months." | No traceable primary source. |
| 24. "9 in 10 projects overrun." | That's megaprojects, on a stated baseline; about half of large projects come in on budget. |
| 25. "Design-build delivers 102% faster." | A square-feet-per-month speed ratio for buildings, not projects finishing in half the time. |
| 26. "$177 billion lost to non-optimal time." | A 2018 vendor survey extrapolation: directional only. |
| 27. "41% of the construction workforce retires by 2031." | Found only in secondary sources; check the primary labour tables first. |
| 28. "The 1-10-100 rule." | Traces to a 1992 book with no data; versions differ. |
| 29. "Rework is 30% of construction." | A report figure; commentators settle near 5% direct. |
| 30. "60–70% of Lean Six Sigma programmes fail." | Old, with varying definitions of failure. |
| 31. "Operations are 80% of lifecycle cost" or "1:5:200". | Couldn't be substantiated when examined; real ratios are lower and vary widely. |
Say instead: "Here's how comparable projects actually went, from named sources, and here's what our own nonconformances cost last year." CM-RSK-02 · QMS-COQ-01 · QMS-COQ-03 · QMS-RWK-03
| Stop quoting | What it actually rests on |
|---|---|
| 32. "AI makes developers 55% faster." | One greenfield lab task. Workplace trials show more modest gains, and one showed experts slowing down. |
| 33. "The 10x programmer." | Mixed conditions and languages; at most about 5 to 1 is down to individuals. |
| 34. "Elite performers deploy 973 times more often." | Year-specific cluster ratios from 2021; the tiers have since been retired. |
| 35. "Only 16% of software projects succeed." | A proprietary method with criticized definitions of success. |
| 36. "Expert personas and 'think step by step' make AI more accurate." | Controlled 2025 tests found little or no gain on current reasoning models. |
| 37. "It passed the engineering exam, so it can do engineering." | An aggregate score hid zero correct answers in structural design. |
| 38. "Grounded AI doesn't hallucinate." | Grounded commercial legal tools were wrong 17 to 33% of the time. |
| 39. "People spend 1.8 hours a day searching for information." | A 2012 extrapolation, repeated mostly by vendors. |
| 40. "Knowledge-Centered Service cuts costs by X%" or "AI deflects X% of tickets". | Member- or vendor-reported; no independent controlled study. |
Say instead: "We'll measure the speed-up with telemetry on our own work, and test the assistant on our own questions before we trust it." SWD-AI-01 · SWD-AI-03 · SWD-MET-02 · PRM-EVD-02 · PRM-EVD-06 · KIN-TWN-04
You don't need a research team to catch most folklore. Five questions do most of the work:
When a number fails these questions, you rarely lose the argument by dropping it. "We don't know the industry rate, so here's ours" is more persuasive to a sceptical executive than a famous statistic they've already heard debunked.
The habits. Don't repeat a statistic you can't source. Keep every number attached to its population, method and year. Prefer your own baseline to an industry average. Treat vendor figures as claims. Say "associated with", not "proven", when that's all the evidence supports.