The Importance of Data Cleansing – The Foundation of AI in the Construction Industry

In previous chapters, we discussed why it's worth exploring AI in the construction industry and what initial steps we can take. Now, let's delve into a crucial topic without which no AI initiative can succeed: data cleansing.

Why Is Data Cleansing Critical to AI Success?

As the saying goes: "garbage in, garbage out." This is especially true for artificial intelligence. According to a McKinsey study, more than 70% of failed AI projects attributed their failure to poor data quality and lack of proper data cleansing. Data cleansing isn't an extra step; it's a prerequisite for effective AI implementation.

Unique Challenges of Construction Industry Data

The construction industry faces particularly significant challenges regarding data quality:

1. Heterogeneous data sources: Blueprints, budgets, schedules, supplier information – all in different formats across different systems.

2. Manual data entry: Many processes still rely on manual data entry, increasing the potential for errors.

3. Varying terminology: The same building element, material, or process may appear under different names in various documents.

4. Incomplete data: Especially common in retrospective documentation, when reconstructing precise facts becomes difficult.

5. Unstructured data: Construction logs, emails, and meeting minutes contain valuable information but not in standardized formats.

Data Cleansing Process Steps in Construction

For construction data to be ready for AI processing, a consistent and thorough data cleansing process is necessary:

1. Data Examination and Evaluation

Before performing any cleansing, it's important to thoroughly understand the available data. This includes:

• Identifying data sources and evaluating their reliability

• Creating statistical summaries (e.g., pivot tables)

• Examining missing values, outliers, and inconsistencies

• Determining basic quality indicators

This step helps gain a comprehensive picture of the dataset's general condition and the necessary cleaning tasks.

2. Applying Data Cleansing Techniques

Particularly effective data cleansing techniques in construction include:

Eliminating Duplications

One of the most common problems is the presence of duplicate data, especially when merging multiple data sources.

Construction example: During plan modifications, the same building element may appear multiple times in records with different versions. Providing elements with unique identifiers and removing duplicates is crucial.

Managing Outliers and Irrelevant Data

Outliers often indicate errors, though sometimes they represent real but unusual data points.

Construction example: If the price of one window type significantly differs from similar windows, it could be a genuine premium product or a data entry error. A combination of industry knowledge and data analysis can help properly handle the outlier.

Fixing Structural Errors

Structural errors include inconsistent naming, spelling mistakes, or formatting inconsistencies.

Construction example: The same room might be labeled differently across documents: "AA.1023", "AA 1023", "1023". These must be standardized before data analysis.

Handling Missing Data

Three main methods exist for handling missing data:

1. Deletion: Removing rows containing incomplete data (only recommended when there are few such rows and the information isn't critical)

2. Imputation: Estimating missing values based on other data (e.g., average costs of similar rooms)

3. Flagging: Informative labeling of missing values (e.g., "no fire rating")

Construction example: In a door schedule, missing values in the "Fire Rating" column might indicate that the particular door has no such rating.

3. Verification

After data cleansing, it's important to check the results:

• Do the cleaned data meet quality expectations?

• Can remaining problems be easily spotted through data visualization?

• Is the new dataset consistent?

4. Documentation and Reporting

Detailed documentation of the cleansing process and operations performed is essential:

• Which data fields were modified?

• What rules were applied?

• Differences between the original and cleaned datasets

• Remaining quality issues or limitations

This helps in reproducing the process and similar processing of other datasets.

Benefits of Data Cleansing for AI Applications in Construction

Clean, consistent, and precisely structured data provide numerous advantages for AI-based solutions in construction:

1. More Accurate Forecasts and Estimates

AI models based on clean data can provide much more accurate estimates for:

• Project duration

• Costs and their variations

• Material requirements

• Resource needs

According to international research, AI models based on more accurate data can reduce cost estimation errors by up to 25%, resulting in significant savings.

2. More Effective Decision Support

Clean data enables:

• Early problem identification

• More accurate risk assessment

• Fact-based, faster decision-making

3. Automated Reporting and Documentation

Standardized, cleaned data enable:

• Automatic report generation

• Acceleration of documentation processes

• Reduction of human errors

4. Improved Project Implementation

Indirect benefits of data cleansing in project execution:

• Fewer delays and redesigns

• More efficient resource allocation

• Better communication between project teams

The data cleansing process and its benefits (Source: Teradata)

Summary: Clean Data Is More Valuable Than We Might Think

Data cleansing isn't just a technical issue but a strategic investment. Clean, reliable data:

• Enable better decisions

• Reduce risks and errors

• Lay the foundation for successful AI projects

• Provide competitive advantage during digital transformation

For construction SMEs, data cleansing can represent the first step toward digitalization – a step that brings immediate benefits while preparing the ground for more advanced technologies, such as AI.

As we mentioned earlier, AI isn't magic but a tool – and like any tool, it's only as good as the quality of the materials used with it. In construction, these materials are none other than data.

What's Next?

If you want to launch your business toward data-driven operations and AI use, here are some specific suggestions:

1. Create a data inventory: Collect all available project data and evaluate their quality.

2. Identify the most valuable data: Which data would help most in decision-making? Which ones experience the most problems?

3. Launch a pilot data cleansing project: Select a smaller but valuable dataset and clean it using the methods described above.

4. Measure the results: Document how processes and decisions improved using cleaned data.

5. Build gradually: After initial successes, extend data cleansing to additional areas while implementing data quality assurance practices.

Data cleansing may not be the most exciting element of digital transformation, but it's definitely one of the most useful – and it creates foundations upon which you can build lasting success.

Sources:

Lexunit: "This is why most AI projects fail" - Comprehensive article on the importance of data cleansing for AI project success. Lexunit (Retrieved: April 10, 2025)

Bim Corner: "Data cleaning - all you have to know" - Detailed guide on the data cleansing process in construction. Bim Corner (Retrieved: April 10, 2025)

Astera Software: "A Comprehensive Guide to Data Cleansing" - Comprehensive presentation of data cleansing techniques and benefits. Astera Software (Retrieved: April 10, 2025)

Alation: "What Is Data Quality and Why Is It Important?" - Detailed analysis of business benefits of data quality. Alation (Retrieved: April 10, 2024)

Forbes: "Building The Future: How AI Is Revolutionizing Construction" - Comprehensive analysis of AI application and benefits in construction. Forbes (Retrieved: April 10, 2024)

Bolpagni, M. & Bartoletti, I. (2021): "Artificial intelligence in the construction industry: adoption, benefits and risks" - Scientific study on the benefits and risks of AI implementation in construction. ResearchGate (Retrieved: April 10, 2024)