If your content is good for the end user, it’s good for every tool along the way. I have proofs.

Let’s talk front-end data; we often call it content. If your front-end data is designed badly so that humans can’t read it, it won’t work well for search engine optimization, answer engine optimization, or generative engine optimization. In computer science (where I started), we use the acronym GIGO; garbage in, garbage out. If you encode bad data into your messages, your audience will decode bad data.

But, can’t we just AI that?

AI is not a verb. It is a noun. It can be a programming paradigm, as in Artificial Intelligence and Machine Learning. It can also refer to an entity or pseudo-entity. If we anthropomorphize an artificial intelligence, which is the mode these days, we refer to Gemini, or Claude, or Perplexity, or ChatGPT. We treat them as if they are humans, and this isn’t necessarily wrong, just strange.

But you cannot ‘AI’ something. You can use an AI to perform an action, or a series of actions, or even create an entire system of actions. If you use an agentic AI, you can execute an entire self-perpetuating series of actions from a well-designed series of prompts. But you cannot take dirty data and ‘just AI it’ into being clean data, as if it is some sort of spell.

Or can you?

I’m currently consulting on a project to use AI to take dirty, nonstandardized data and put those data (FYI: Data is a plural word. Datum is singular.) into a clean-ish, largely AI-readable state. Right off the bat, I don’t think this is the right way to do it, but let’s follow my standard methodology and take this problem apart. Here’s the problem statement:

Problem: We can’t get reliable results from our data when we ask AI to parse it, or we try to parse it ourselves, because it is in a series of non-standard formats.

The parts of the problem

Building things with blocks

Illustrations

Building things with blocks

Built with Firefly

Following my methodology, we can take this problem apart and look at the individual parts, then find a strategic way to attack the problem.

  1. We cannot
  2. Reliable results
  3. Our data
  4. AI parsing
  5. Manual parsing
  6. Data exists in a series of non-standard formats
  7. Negative satisfaction and emotions from our results

Following my methodology, we should look at the parts of the problem, find the weakest points, and then engineer a solution that will cause a cascade failure in the rest of the problem.

If this doesn’t sound like how a communications professional would approach a problem, you’re probably right; I don’t do that. I don’t look at problems and diagnose them by how many witty tweets and press releases they need. I examine problems, find out where we can strategically apply leverage, and cause cascade failures in problems in such a way that we can replace the problem with a functional system, often using the same parts. I have decades of diverse experience in using many approaches to solve problems, so that’s what I do. That, and the professors at Oregon State are some of the best in the world at explaining communications theory (big props to Professor Dan and Dr. Reeves) so I have a whole library of tools at my disposal.

Looking at the parts of the problem in series, we have our inability, our satisfaction with the results (2 and 7), the state of our data, the ability of AI to parse our data, whether we can manually parse the data, the format of our data, and our respective emotional satisfaction in our data.

Here’s where things diverge;

  • I believe that the best way to get from dirty data to clean data is to clean and reformat your data into a universally acceptable format so it can be moved from system to system, parsed by AI, read by humans, and get good SEO & SERP results (SEO is Search Engine Optimization, SERP is Search Engine Results Page).
  • My counterpart believes that we can take dirty front-end data and get an AI to read it, and then output it into a format that an AI can read.

We’ll look at each argument in turn before examining the whole method and all the pieces. I go first because I’m writing this.

Update

A separate client that I consult for occasionally has had a series of meetings about her data. She runs an e-commerce operation, and a lot of her data is siloed and in different formats. Various businesses would love to sell her company services in which they take existing data, cast an AI spell on it, and it becomes unified, clean data! Except, to quote;

“Data must exist and be available.”

If your data doesn’t have the attributes it needs for the destination format - tags, categories, schema, whatever - you absolutely must add those in, first. Asking an AI to do that for you is literally asking an AI to fake your data. Don’t do that. It isn’t magic. It’s work, plain and simple. You need your data, and the following section will tell you why.

Clean your data first

Sub-thesis: If you take your existing dirty data and put it into a human-readable format in a system that is machine-readable and transferable you’ve solved the entire problem.

Following my own methodology, you design a solution that attacks 3, 4, 5 and 6. This causes a cascade failure in the problem, which then allows us to solve the entire problem! The cost? We have to take the time to fix our data.

If you don’t follow this method, however, you’ll rack up technical debt; and that will need to be paid someday. Look at all of the organizations that refused to get off of COBOL; they’re having to fix that now, two and a half decades after Y2K.

The logos proof is that if you design your data in a way that it can be used by the end user, it will be usable and parseable by every tool along the way, You just have to understand that AI tools are, in fact, tools. They are not witches, they are not gods. They are the products of businesses, and as the products of businesses there are certain rules they have to adhere to;

  1. Businesses have to make money or
  2. Businesses need to make a thing that their clients pay them money for, or
  3. If they don’t make money, they need to provide something of value to their clients.

Any tool that can survive as a business in the ecosystem between the data and the end user of the data has to be usable by the end user, because otherwise it would render the data unusable. Therefore, if you present it with clean data, it can only make the data better! An AI that can present your data in a neat and shiny format works best with clean data, Garbage in, garbage out. Shiny in, shiny out.

Nah, don’t bother cleaning it, just make AI do it

This is a method that a lot of organizations are trying to follow, and I’ll walk you through without judgement first before talking about why I think this is a bad idea.

My counterpart is trying to take 1,700 documents that his organization has - in a variety of formats and levels of sophistication - and generate a semantic knowledge graph from it; one that humans, via search, AI, via scraping, and search engines, via scraping, would be able to access. This part of his method, I applaud. I think it’s brilliant. It is a middle-of-the-road solution that doesn’t require you to clean your data.

In a perfect universe, he would have a library of 1,700 PDFs, written with either actual text or OCR-scrapeable images, that he could then use an AI to scrape. And because many AI use a multi-modal scraping engine, they could probably even scrape non-accessible text; if you wrote a document on your windows 3.1 machine back in 1994 that is completely unreadable by normal OCR methods, maybe they could scrape your document and make it make sense. That would be so cool.

The problem isn’t necessarily that he has 1,700 PDFs to scrape, it’s how things were deposited and evolved over time. The org has been depositing documents on the website since 1995, and has migrated through several CMS (content management systems) in the intervening decades. But not every document moves cleanly from repository to repository.

Why ‘AI-ing’ things doesn’t work, and why it’s actually beneficial to clean your data

The average IC (individual contributor) stays in their role for 3.9 to 4.8 years, which means that, since 1995, my counterpart’s organization has seen, on average, 6 to 7 individual contributors who control the strategy of how those documents are stored. Let’s be generous and say that, in 31 years, this organization has seen one strategist every 5 years; so a total of 6 strategists. That means 6 different approaches to storage, file pathing, and naming.

How much do you want to bet that there are no documents in dark corners that store valuable data? Are you willing to bet your business on that?

This becomes very important if you work in medical, legal, or financial sectors, but it’s important for a lot of us. If you don’t provide those documents to your AI, you won’t have access to it in the final product.

Okay, but let’s talk about the benefits. I’m a big fan of not using fear appeals. Decades of climate change communication have taught us that fear appeals don’t work; so let’s talk about what’s in it for you, because that’s the real motivating factor. You want to know;

  • How can I make money off of cleaning my data instead of ‘AI-ing’ my data? or
  • What data can I use to convince my boss to let me invest in cleaning my data first?

We’re working off one use case - a company has a ton of legally poignant documents in a myriad of formats and in a ton of locations that they’ve stowed things in for the last two to three decades. There are other use cases; I’ve worked with a company that was trying to migrate its financial records from one Magento instance to another Magento instance, and they never stored their financial records in some sort of financial record-keeping system. Another company stored half of their legal records on Sharepoint and half of their legal records on external hard drives, with no central data storage methodology at all.

For this one use case, however, the best way to go would be to manually clean or migrate the data. If you don’t have the manpower to do it, there are agencies that would be more than happy to do so for money. But cleaning your data is a great opportunity to conduct a content audit; look through everything to see what is redundant, old, or trivial. The common acronym is ROT; perform a ROT analysis.

When you clean your data, you get the opportunity to get rid of data that’s costing you storage space. Also, you can put your data into a usable format that will be migrateable in the future. I recommend some sort of database, possibly a relational database, where your documents are all stored in a Unicode-encoded format. Very readable and future-proof for the near future, or at least the next 30 to 40 years. Beyond that, you’ll need to re-migrate, so you may as well plan for that now. Formats (and databases) change over time. Just look at COBOL; nothing lasts forever.

Why AI-VS-User isn’t a real thing

My boss’s boss has this idea, and I’ve noticed it in a lot of people, that you can either make content ‘good for AI’ or good for people, and that’s an A/B problem. This is a false dilemma.

If you spend any time reading Google’s SEO documentation you’ll see that content that is;

  1. Good for SEO
  2. Is also good for humans

-and it isn’t different for AI-optimized content. There are a few things you need to do for SEO, AEO and GEO that you don’t need to do for human readability, but not a lot;

  • Add all the meta stuffs (not META the company, but meta as in metadata) to your pages
  • Ensure your schema tags make sense (this is also metadata, but a slightly different flavor)
  • Add everything necessary for good OG (opengraph) results because everyone likes rich cards
  • Keep a weather eye out for updates, in case someone comes up with a connection scheme that AI uses which search engines don’t use (that’s highly unlikely; Google has been using AI in their search since 1999, but keep ya eyes open)

And here’s the real rub, which should probably be top matter; AI and SEO are designed to find the content that is most usable for humans.

Let me say that again.

AI and SEO are designed to find the content that is most usable for humans.

AI, SEO, GEO, whatever; the companies want to make money. And they make money by serving the right content to humans. There is no SEO hacking. There is no AEO or GEO hacking. In fact, search engines and other AI products get better over time, so there will be even fewer ways that organizations can manipulate SEO, AEO and GEO results as time goes on. To truly future-proof your content;

  • Write for clarity.
  • Put your content into one standard format.
  • Leave documents for humans in the future.