CRISP-DM (Cross-Industry Standard Process for Data Mining)
1. Overview
A. Definition
CRISP-DM (Cross-Industry Standard Process for Data Mining) is a cross-industry, tool-independent standard process model that structures data mining and data analytics projects into six phases: Business Understanding → Data Understanding → Data Preparation → Modeling → Evaluation → Deployment. Beginning as a European ESPRIT project in 1996, it was codified as CRISP-DM 1.0 in 1999–2000 by a consortium of SPSS (ISL), NCR, Daimler-Benz, and OHRA, and its core is a structure with feedback loops that return to earlier phases and an overall cycle that loops from deployment back into business.
The typical reason a data mining project fails is not a shortage of algorithms but diving into data and models without defining the business problem to be solved. Even a dazzling model has no value if it is never used in a frontline decision; conversely, if data quality is poor, no model can be trusted. CRISP-DM nails down, as a process, the view that "analysis starts from a business question, not from data, and ends with operational deployment, not with a model," so as to avoid this trap.
CRISP-DM matters because it is not a mere task checklist but a shared language for analytics projects and a quality management framework. When teams across different organizations, industries, and tools share the vocabulary of CRISP-DM's six phases and deliverables, they can communicate progress against the same criteria and surface risks early by checking each phase's completion conditions (gates). From a professional engineer's perspective, CRISP-DM is, alongside KDD and SEMMA, the de facto standard of data analysis methodology and the starting point for the data analysis lifecycle thinking that leads to today's MLOps and CRISP-ML(Q).
B. Background and Necessity
The mid-to-late 1990s was a time when data warehouses and statistical/machine learning tools spread rapidly and firms sought to mine value from accumulated data in a "data mining" boom. Yet projects at the time relied excessively on an individual analyst's intuition and tool-handling skill, and even within the same company approaches differed from team to team, so there was no common procedure to guarantee reproducibility and quality. In particular, it was common for frontline users to distrust analytic results, or for hard-won models to be shelved without ever being deployed into real systems.
CRISP-DM arose from this awareness, aiming for a procedure that is "reproducible no matter who performs it, applicable in any industry, and not locked to any particular vendor tool." Its necessity has only grown as data has grown in scale and scope of use. In the big-data/AI era, analytics projects involve diverse stakeholders—data engineers, analysts, business, security, operations—so a common frame that clarifies per-phase responsibilities and deliverables is a precondition for collaboration. Indeed, in surveys a majority of data scientists still cite CRISP-DM as the most widely used analytics methodology, so even more than two decades on it functions as the basic skeleton in the field. In short, CRISP-DM is an old concept whose practicality grows again as data-centric organizations multiply.
2. Overall Structure and Cyclicity
The crux of CRISP-DM's overall structure is that it is not a one-directional waterfall of six phases but an iterative process that frequently returns between phases and cycles as a whole. The diagram below shows the six phases and the major feedback flows between them. The outermost cyclic arrow indicates that even when one project ends, its results spawn new business questions that carry into the next cycle.
flowchart TB
BU["1. Business Understanding<br/>(define goals & success criteria)"]
DU["2. Data Understanding<br/>(collect, explore, quality check)"]
DP["3. Data Preparation<br/>(cleanse, transform, features)"]
MO["4. Modeling<br/>(select technique, train)"]
EV["5. Evaluation<br/>(against business goals)"]
DEP["6. Deployment<br/>(release, operate, monitor)"]
BU <--> DU
DU --> DP
DP --> MO
MO <--> DP
MO --> EV
EV --> DEP
EV -->|redefine if goals unmet| BU
DEP -->|new questions, degradation| BU
style BU fill:#e8f0fe,stroke:#2f6fed,stroke-width:2px
style EV fill:#fef7e0,stroke:#f29900,stroke-width:2px
style DEP fill:#e6f4ea,stroke:#188038,stroke-width:2px
The purpose and representative deliverables of the six phases are first summarized in the table below. This table is only a skeleton, however; why each phase sits in its place and what it decides is described in the following sections.
| Phase | Core question | Main activities | Representative deliverables |
|---|---|---|---|
| 1. Business Understanding | What and why | define goals, success criteria, constraints | project plan, analysis goals |
| 2. Data Understanding | What data exists | collect, descriptive stats, quality diagnosis | data exploration report |
| 3. Data Preparation | How to make it analyzable | cleanse, integrate, transform, features | final analysis dataset |
| 4. Modeling | Which technique to use | select algorithm, train, tune | trained model, parameters |
| 5. Evaluation | Did it meet business goals | performance, validity, review | evaluation result, approval |
| 6. Deployment | How to turn it into value | deploy, monitor, document | operational model, maintenance plan |
Understanding CRISP-DM's non-linearity is the essence of the methodology. For instance, if quality problems surface during Data Understanding, one must return to Business Understanding and adjust the goal itself; if performance falls short during Modeling, one returns to Data Preparation and rebuilds features. Moreover, the literature reports that a substantial share of actual project effort (often 50–70% or more) concentrates in the Data Preparation phase, which shows that modeling is not the whole of analysis but holds only atop well-prepared data. CRISP-DM is field-friendly in that it acknowledges this reality through its cyclic structure.
3. The Six Phases in Detail
The following diagram unfolds the representative sub-tasks within the six phases to show the process at the task level. Each phase divides, under its higher goal, into sub-tasks that produce concrete deliverables.
flowchart LR
subgraph P1["1. Business Understanding"]
A1["business goals"] --> A2["situation assessment"] --> A3["convert to analysis goals"] --> A4["project plan"]
end
subgraph P2["2. Data Understanding"]
B1["initial collection"] --> B2["describe data"] --> B3["explore data"] --> B4["verify quality"]
end
subgraph P3["3. Data Preparation"]
C1["select"] --> C2["cleanse"] --> C3["construct & features"] --> C4["integrate & format"]
end
P1 --> P2 --> P3
style P1 fill:#e8f0fe,stroke:#2f6fed,stroke-width:2px
style P3 fill:#e6f4ea,stroke:#188038,stroke-width:2px
A. Business Understanding
The first phase is not analysis but problem definition. The business goal (e.g., "reduce customer churn") is translated into a data mining goal (e.g., "predict next-month churn probability and select the top 10%"), and success criteria are nailed down on two sides: business metrics (a 2%p drop in churn) and technical metrics (AUC ≥ 0.8). If business, analytics, and operations do not share the same goal in this phase, then however good the later model, it cannot answer the question "so what is it for?"
Available resources, constraints, and risks are also assessed to form a project plan. Constraints such as data access rights, privacy regulation, deadlines, and the operating environment are hard to change the later they are addressed, so they must be identified early. For a bank's credit-scoring model, for instance, explainability and regulatory compliance become success criteria as important as performance, and this constrains the choice of modeling technique (an interpretable model rather than a complex black box) from the very start.
B. Data Understanding
In the second phase, the actual data is collected and examined exploratorily (EDA). Which tables/fields exist; what the distributions, missingness, outliers, and label imbalance look like; and whether the data fits the business hypothesis are checked with descriptive statistics and visualization. This process often reveals facts like "the desired variable does not exist at all" or "the labels are not trustworthy," in which case one must return to Business Understanding and adjust the goal to reality (feedback loop).
Data Understanding concludes with verifying data quality. Missing rates, duplicates, value consistency, and biases at the time of collection are diagnosed and documented. For example, in telecom churn prediction, if the "cancellation date" field is recorded differently in the marketing system and the billing system, modeling without catching this discrepancy leads to data leakage—future information seeping in—so that evaluation performance comes out unrealistically high. Catching such traps early is the value of this phase.
C. Data Preparation
The third phase is the process of processing source data into the final dataset the model will learn from and, as noted, takes the most effort in the project. It consists of sub-tasks: selection (narrowing to needed records/fields), cleansing (imputing missing values, handling outliers), construction (derived variables, feature engineering), integration (combining multiple sources), and format conversion. Feature engineering in particular governs model performance; a derived variable such as "rate of change in call volume over the last three months," for instance, often explains churn far better than raw call volume.
Data Preparation forms the strongest feedback loop with Modeling. If model performance is insufficient, one creates new features or changes scaling/encoding methods and prepares again. Moreover, every transformation in this phase must be reproducible and applicable identically in operation. If the feature-generation logic differs between training and deployment (training-serving skew), real-world performance plummets, so managing the transformation pipeline as code is the modern best practice.
D. Modeling
In the fourth phase, algorithms are finally applied. A technique suited to the problem type (classification, regression, clustering, association) is chosen, training/validation data are split, the model is trained, and hyperparameters are tuned. The key point is that technique selection is already constrained not in the Modeling phase but in Business Understanding. If explainability is required, one picks logistic regression or decision trees; if top performance is the priority, gradient boosting or neural networks.
Modeling frequently takes the loop back to Data Preparation. Some algorithms do not tolerate missing values or require particular scaling, so changing the technique also changes data preparation. In addition, overfitting is checked with cross-validation and several candidate models are compared to select the one to pass to the next, Evaluation phase. Here the performance metric should be viewed not as a single accuracy but as a metric fitting the business context (for imbalanced data, precision, recall, AUC).
E. Evaluation
The fifth phase is a gate that judges not the model's technical performance but whether the business goal has been achieved. Against the success criteria defined in phase 1 (churn reduction, AUC threshold), one examines whether the model is usable, whether any overlooked business issues remain, and whether there are regulatory or ethical risks. Even with good technical metrics, if the business effect is uncertain the decision may be not to deploy; conversely, if the goal is unmet, one returns to Business Understanding and runs the cycle again.
It is important to distinguish Evaluation from the evaluation within Modeling (performance measurement inside phase 4). Where the in-modeling evaluation looks at "are the metric numbers good," the phase-5 Evaluation renders a go/no-go approval on "is this model qualified to enter real decision-making?" For example, if a loan-screening model has high AUC but is found to be biased against a particular group, deployment is suspended over fairness and it returns to data preparation/modeling redesign.
F. Deployment
The final phase deploys the approved model into actual business and systems to turn it into value and continuously manages the results. Deployment is not merely uploading a model file; it connects predictions to business processes (campaign-target selection, a real-time recommendation API, etc.) and includes operational monitoring, a retraining plan, documentation, and handover. This is why CRISP-DM's Deployment phase explicitly stresses "organizing data mining results into a form the organization can use."
Deployment is not the end of the project but the start of the next cycle. A deployed model degrades (model/data drift) as the data distribution changes over time, so when monitoring detects deterioration, one returns to Business Understanding to re-examine the goal and retrain. This very point is where CRISP-DM meets modern MLOps, treated in the deep-dive below.
4. Comparison — Differences from KDD and SEMMA
CRISP-DM is the most widely used among data analysis methodologies but not the only one. Representative others are KDD (Knowledge Discovery in Databases) and SAS's SEMMA (Sample-Explore-Modify-Model-Assess). The difference among the three lies not in the mere number of phases but in "where it starts and where it ends"—that is, whether business and deployment are included in the process.
| Aspect | CRISP-DM | KDD | SEMMA |
|---|---|---|---|
| Proposed by | industry consortium (1999) | Fayyad et al., academia (1996) | SAS Institute |
| Number of phases | 6 phases | 5–9 (select, preprocess, transform, mine, interpret) | 5 phases |
| Business understanding | explicitly included | weak (starts at data selection) | none (starts at sampling) |
| Deployment/operation | explicitly included | interpretation/use centric | none (ends at assessment) |
| Orientation | solving the business problem | knowledge discovery (academic) | tool (SAS)-centric analysis |
The reason the key difference arises lies in each methodology's birth context. KDD systematizes the academic process of "discovering knowledge" from data, so it weights data processing and pattern interpretation, and business goal definition or operational deployment is relatively weak. SEMMA reflects the analytic workflow of the SAS Enterprise Miner tool, concentrating on "modeling practice" from sampling to assessment, so problem definition and deployment are out of scope. CRISP-DM, by contrast, was made out of the field demand to "make analysis actually used in decisions," so it places business understanding and deployment at both ends and cycles the whole. The practical implication is clear—if the purpose of analysis is not "presentation-ready insight" but "a system deployed to the field," CRISP-DM's scope fits best.
As a concrete case, a retailer ran a demand-forecasting project in KDD style and obtained a sophisticated forecasting model, but omitted the deployment design to connect it to the ordering system, so the field still placed orders manually by Excel. Had they explicitly walked through CRISP-DM's Deployment phase, they would have designed from the start "how to feed forecast results into the ordering process." Conversely, SEMMA is highly productive when running iterative model experiments quickly in a SAS environment, so the three methodologies are not exclusive but selected and combined by situation.
5. Deep-Dive — Evolution into CRISP-ML(Q) and MLOps
CRISP-DM, designed on the premise of traditional data mining, has been noted to lack detail after deployment (maintenance and monitoring) for modern machine learning services that are continuously retrained and operated. A representative extension aiming to remedy this is CRISP-ML(Q) (Cross-Industry Standard Process for Machine Learning with Quality Assurance), proposed by Studer et al. in 2021. The diagram below shows the CRISP-ML(Q) lifecycle.
flowchart LR
S1["Business & Data Understanding<br/>(merged phase)"]
S2["Data Preparation"]
S3["Modeling"]
S4["Evaluation"]
S5["Deployment"]
S6["Monitoring & Maintenance"]
S1 --> S2 --> S3 --> S4 --> S5 --> S6
S6 -->|detect drift, degradation| S1
style S1 fill:#e8f0fe,stroke:#2f6fed,stroke-width:2px
style S6 fill:#fce8e6,stroke:#d93025,stroke-width:2px
CRISP-ML(Q)'s changes can be summarized in three points. First, it merged CRISP-DM's Business Understanding and Data Understanding into a single phase. In practice, "what to solve" and "what data exists" are not separate but strongly intertwined. Second, it added a Monitoring and Maintenance phase after deployment. It elevates the phenomenon of a model degrading as the environment changes during operation into a formal phase, treating performance monitoring, retraining triggers, and rollback as part of the lifecycle. Third, as the (Q) in the name indicates, it defines a risk-based quality assurance (QA) methodology for each phase. It identifies ML-specific risks that can arise in each task (data leakage, distribution change, lack of reproducibility, etc.) and specifies the checking activities that respond to them, raising analysis to trustworthy engineering.
This evolution naturally connects to MLOps (Machine Learning Operations). MLOps realizes the lifecycle defined by CRISP-DM/CRISP-ML(Q) with CI/CD, automation, and monitoring tools, turning everything from data collection through training, deployment, and retraining into a reproducible pipeline. That is, if CRISP-DM is the methodological skeleton of "what to do in what order," MLOps is the execution system of "how to automate and operate it." For instance, when a recommendation-system service detects a drop in click-through rate at CRISP-ML(Q)'s Monitoring phase, the MLOps pipeline automatically triggers retraining, validates via A/B testing, and then deploys. In the generative-AI/LLM era, data and model governance and prompt/evaluation (LLMOps) elements are added, making the lifecycle even more refined.
6. Considerations and Implications
The points to note from a professional engineer's perspective when applying CRISP-DM to real analytics projects and organizational operation are as follows.
Run the methodology as iteration and gates, not as a sequence. CRISP-DM's value lies not in memorizing six phases but in placing completion conditions (gates) at each phase and returning to an earlier phase when needed. Applying it one-directionally like a waterfall discovers data quality problems or goal misalignment late, inflating rework cost. An operating rule that specifies deliverables and entry criteria for the next phase at each phase's close is needed.
Invest sufficiently in Business Understanding and Deployment. Analysts want to pour time into modeling, but a project's success or failure is decided at the two ends (problem definition and deployment). Define goals and success criteria doubly as business metrics and technical metrics, and bring monitoring, retraining, and handover after deployment into scope from the start so that analysis is not shelved. In particular, "presentation-only analysis" that omits the Deployment phase has almost no effect for the investment.
Combine Data Preparation with reproducibility and governance. Data preparation, which takes most of the effort, is the source of quality and reproducibility. Manage transformation logic as code/pipelines to prevent training-serving skew and data leakage, and proactively reflect privacy and regulatory compliance in the Data Understanding/Preparation phases. This connects directly to data governance and data quality management frameworks.
Extend to the modern ML lifecycle and manage trade-offs. Since traditional CRISP-DM is thin after deployment, continuously operated services must combine CRISP-ML(Q)'s monitoring/QA with MLOps automation. However, strengthening automation and governance can conflict with agility, so it is desirable to adjust the level of control in proportion to the project's risk (regulatory, safety, financial impact). A differentiated strategy—lightweight application for small-scale exploration, strict QA for mission-critical services—is realistic.
Outlook: a standard skeleton in the era of generative AI and governance. As data/AI regulation (fairness, explainability, AI management systems) tightens, the importance of a process that makes the whole course of analysis traceable and verifiable grows. CRISP-DM and its extensions will continue to function as the common skeleton on which LLMOps and AI governance frameworks stand, and are expected to be used as a reference frame for assessing an analytics organization's maturity.
References
- CRISP-DM, Wikipedia: https://en.wikipedia.org/wiki/Cross-industry_standard_process_for_data_mining
- Studer et al., "Towards CRISP-ML(Q): A Machine Learning Process Model with Quality Assurance Methodology", Machine Learning and Knowledge Extraction, 2021: https://www.mdpi.com/2504-4990/3/2/20
- IBM, "What is CRISP-DM?": https://www.ibm.com/think/topics/crisp-dm
In one line: CRISP-DM structures data analytics projects into the six phases of Business Understanding, Data Understanding, Data Preparation, Modeling, Evaluation, and Deployment, with inter-phase feedback and an overall cycle at its core; it is a cross-industry, tool-independent standard process that—by including both ends of problem definition and deployment—is distinguished from KDD and SEMMA, and today extends into CRISP-ML(Q) and MLOps as the common skeleton of a trustworthy machine learning lifecycle.