Knowledge With an Asterisk – Why using AI is hard for organisations

We’re entering the era of “knowledge with an asterisk.” The question for organisations is when that asterisk is acceptable.

Generative AI is already excellent at producing fluent drafts such as emails, code, summaries and policy text. But inside organisations that fluency comes with an asterisk: outputs are probabilistic, not reliably repeatable, and therefore harder to defend in audits, courts, or public scrutiny. The practical question isn’t “Can we use it?” but “When is knowledge-with-an-asterisk acceptable?”

This Christmas, between baking logistics and the quiet moral pressure to be “present,” I built and deployed a handful of small apps that now gently shepherd my personal workflow along. Not “apps” in the heroic sense. There were no venture capital pitch and no product launch, just small digital contraptions that do what I keep forgetting or am too lazy to do. A button that turns a messy folder of notes into something searchable. A tiny interface that nudges a recurring task along. A script to extract notes from podcasts I listen to that used to be a fantasy project, now assembled in an evening, mostly by talking to a system that politely pretends it understands me and cheer me on.

Unsurprisingly to anyone who have opened social media in 2025 and been exposed to hundreds of such stories of “vibe coding”, I did it without much coding skill. I mean, I can read code much the way I can read a recipe in a language I don’t quite speak. So, enough to not poison anyone. But these days you can get amazingly far with a mixture of curiosity, stubbornness, and a language model that will happily generate code on demand. And the leap from my little Christmas hobbyism to the big picture is not subtle. If I can duct-tape together bespoke workflow-automation while half-paying attention to a pot of potatoes, then of course generative AI will reshape the world, right?

If we listen to what technologists and the people who are investing hundreds of billions of dollars in compute capacity tell us, then all kinds of work and organizing, science and R&D, and all the bureaucratic middle zones of modern life where most of us spend our hours, will be soon be infused with agentic super intelligence on demand. And yet, while it’s easy to see the affordances and marvel at the potential, it’s often harder to see the value materialising inside organisations.

That’s at least the tension I keep encountering when I talk to practitioners in ordinary firms far from Silicon Valley, or in public administration. Many are at a loss. They know the tools are powerful and they have seen the demos. They’ve been told, repeatedly, that this is the dawn of something like Artificial General Intelligence (AGI), or at least the early misty morning before it. Yet when you ask what they can actually do with it, in the places where the work is messy and consequential and accountable, they shrug. Or they tell you about expensive enterprise licenses for language models with abysmal use numbers.

We are living through a period in which the story about the technology is almost more forceful than the technology itself. Not because the tools are trivial (they certainly aren’t), but because the surrounding mythology is operating on a different scale than everyday practice. In one register, we get the cosmic language: intelligence, consciousness, species-level transformation, the end of work, the beginning of whatever comes after. In another, we get the mundane reality: someone in a municipal office asking whether it is okay to paste a citizen’s email into a chatbot, or a team of developers arguing about whether the generated code is maintainable, or a manager wondering how to measure productivity when output has become cheap but also attached with an asterisk of trustworthiness.

Somewhere between these registers sits the most revealing tension of the moment: organisations are suckers for determinism, or the certainty of outcomes. They are at the very least aspiring for it. Budgets, compliance regimes, procurement rules, safety protocols, audit trails, these are all machines for turning uncertainty into defensible decisions. Bureaucracy, in the Weberian sense, is not merely paperwork; it’s an institutional technology for making actions legible and repeatable. Firms do a similar thing through standardisation, targets, templates, systems, and “best practices.” Not because managers are boring, but because organisations have to survive scrutiny by regulators, customers, journalists, and courts.

Automation, in the classic sense, is appealing precisely because it is supposed to be repeatable: press the button, get the same result every time, and if something goes wrong you can point to a specification and say, “We followed the procedure.”

Generative AI, meanwhile, is not deterministic in that comforting way. It is, at its core, a probability engine with good manners. A large language model (LLM) is trained on massive amounts of text to learn statistical patterns of what words and phrases tend to follow others in different contexts. When you write a prompt, the model doesn’t “retrieve the correct answer” the way a database does. It predicts the next token (roughly: a chunk of text), then the next, and the next, producing a plausible continuation. It produces outputs that are always conditional: conditional on the prompt, the context window, the training data’s odd shadows, the model’s temperature setting (how much randomness you allow), the unseen guardrails, the vendor’s latest update. Even when the output looks crisp, it is still conditional knowing. Behind the curtain generative AI ”simply” predicts the next token based on patterns in its training data. That’s why models can be brilliant one moment and confidently wrong the next. The knowledge produced by GenAI is always knowledge with an asterisk.

That’s a tolerable quirk when you’re using it for brainstorming at home. In organisations it’s something else because determinism isn’t just aesthetic preference, it’s how responsibility travels. Procedures, audit trails, and “what we can defend in retrospect” all depend on the promise that the same input yields the same output, or at least that deviations can be explained. It’s not that organisations are overly careful. It’s that they are built to metabolise certainty, or at least the appearance of it. In other words, the draft is easy, but the decision is what must be defensible.

To be fair, many organisations have been living with this gap for a long time. The fantasy of clean automation, perfectly specified tasks executed by obedient systems, has always collided with the fact that workplaces are not sterile laboratories and that reality tends to be messy. It’s not new then that organisations must cope with uncertainty and unruly technology, but generative AI intensifies the mismatch because it operates in the same medium as so much organisational life happens through, namely language. It writes the email, the memo, the policy draft, the case summary, the code comment, and the meeting minutes. It gives you the feeling that the cognitive part is done, which is very comforting for our attention-impoverished minds. Then it gently hands you a new problem: what do you do with an output that is persuasive but not necessarily true?

Generative AI arrives like a mischievous new clerk who is very fast, rarely sleeps, and has read everything on the cloud storage, plus the entire internet, plus a pile of documents no one would admit to owning. It can produce competent-looking text at a volume that makes traditional throughput metrics look quaint. But it cannot, by itself, participate in the accountability structures that give organisational decisions their legitimacy. It cannot be cross-examined and more importantly, it cannot be ethically blamed in a way that satisfies anyone except perhaps the procurement department.

Consider municipal social services, where the work is intensely linguistic, but also intensely situated. A caseworker reads an application. They interpret a narrative. They document decisions. They translate policy into action under time pressure, with incomplete information, and with serious consequences for the person on the other side of the process. The organisation wants consistency and legal defensibility. The worker wants to do the right thing and survive the workload.

Generative AI can help here in obvious ways such as summarizing case notes, drafting letters, translating bureaucratic language into something more humane, suggesting checklists, even surfacing relevant policy passages. It can reduce the sheer friction of writing. But it can also introduce a dangerous smoothness. A language model may confidently produce an interpretation that sounds reasonable but subtly misstates a rule, or invents a detail, or frames a person’s situation in a way that carries bias. In an environment that depends on auditability, the probabilistic nature of the output becomes not just a technical issue but an organisational risk. The question isn’t “Can we use it?” but “Under what conditions is knowledge with an asterisk acceptable?” When does a draft become a decision? Who must sign it? What counts as a check? What happens when the model’s output is persuasive enough that the human feels redundant?

Now contrast that with the paradigmatic use case for AI, software development, where generative AI has been adopted with almost gleeful intensity. Developers use it to generate boilerplate code, write tests, refactor functions, explain unfamiliar codebases, and scaffold prototypes. The work is still situated, but the feedback loops are different. Code can be run and it can fail loudly. In many cases, the environment provides a kind of determinism that bureaucratic decision-making lacks since the program either compiles or it doesn’t. The model’s probabilistic suggestions can be filtered through tooling, version control, peer review, and automated tests. There is still risk, security vulnerabilities, subtle logic errors, and creeping complexity, but the genre of work is already built around iterative checking and repair.

Many other organisational domains don’t have equivalent feedback systems. Their “tests” are human lives, legal disputes, public trust, or reputational damage. Their bugs don’t crash; they accumulate, silently, and become tomorrow’s scandal.

In other words: software development benefits not because developers are more enlightened, but because the socio-technical arrangement makes verification tractable.

(Organisation scholars would call this a reminder that technologies don’t have “impacts” in the abstract; they become what they become through practice, infrastructure, and accountability arrangements. What Wanda Orlikowski and Susan Scott would frame as sociomaterial entanglement, and what Lucy Suchman would call the situated character of action.)

This is one reason the “AGI” story feels so slippery. In some contexts, probabilistic assistance slots neatly into existing practices. In others, it collides head-on with institutional demands for traceability, stability, and justification. The same underlying technology becomes different things depending on the arrangement around it: what data it can touch, what tools it is connected to, what review practices exist, what liabilities are at stake, what metrics managers care about.

And then there is the infrastructural reality that the more organisations depend on these systems, the more they depend on whoever controls the model access, the compute, and the pricing tiers. If a municipality adopts a generative AI layer for drafting and triage, it also adopts an external dependency that can change terms, alter behaviour through updates, or simply become unaffordable. If a research group builds workflows around a particular model’s quirks, they inherit the vendor’s roadmap as a constraint on their own practice. We talk about “capabilities,” but the mundane governance of access and dependence may be the more consequential story.

There is also something quietly subversive about what generative AI is best at. Not the grand organisational “transformation” decks, but the small, bespoke contraptions that fit one person’s odd little workflow like a well-worn shoe. My Christmas apps were basically that: personal infrastructure, built around my own irritations, preferences, and half-formed routines. This is where the technology shines because the marginal cost of making a bespoke digital tool has collapsed, and because language is suddenly a usable interface to computation. You do not need to convince an IT department, a procurement process, or a steering committee. You just need a problem you can describe, and the patience to iterate until the tool behaves. That should be exciting for organisations, but it can also be deeply uncomfortable.

Historically, managers have had an ambivalent relationship with worker know-how. Before mechanisation and scientific management, many crafts were organised around skill that sat in people rather than in procedures, and that gave workers a particular kind of power: they could pace the work, guard the tricks of the trade, and resist being easily replaced. A large part of industrial modernity, as labour process scholars have long noted, involved extracting that situated knowledge from workers and re-embedding it into machines, workflows, and managerial systems, so that the organisation could own the method as well as the output. Generative AI, at least in its everyday form, threatens to reverse the direction of travel. It equips each employee with a kind of personal micro-factory for text, code, and process glue, but one that is hard to standardise, hard to audit, and often impossible to see from the outside. You can almost hear the ghost of Frederick Taylor clearing his throat in the background, asking where the “one best way” has gone, while the workforce effortlessly builds a hundred small ways that work well enough.

So, what then, should the certainty-craving organisation do to make use of the very real potential of generative AI?

First, stop asking “Where can we automate?” at a grand scale and instead ask “How can we empower employees safely?” The easiest win is not enterprise level decision-automation. It’s bespoke and small-scale decision support and automation where drafts are helpful and errors are cheap. Rather than engaging in total transformation, ask in each setting: what are we trying to achieve, what risks are we willing to carry, and what practices will make this tool accountable to the people affected by its use?

Sometimes knowledge with an asterisk is fine, such as drafting code suggestions in a sandbox, brainstorming research questions, or generating multiple phrasings for a paragraph. Sometimes it might not be, like automated decisions about welfare eligibility, legal judgments, disciplinary actions, or medical triage.

Second, treat verification as real work. If you adopt AI, you are adopting a new kind of work consisting of legitimatising the AI output. There is a small irony here as the grand promise of AI is often framed as a future in which machines will finally do “the hard cognitive stuff,” freeing humans for more meaningful pursuits. Yet the more immediate reality is that humans are being asked to do a different kind of hard work which is the work of judgment under uncertainty, the work of verification, the work of deciding when a fluent artifact is trustworthy enough to act on. That labour is often what Lise Justesen and Ursula Plesner calls invisible digi-work, the kind of work that quietly makes systems function and is often taken for granted. If you don’t allocate time, training, and status to it, you’ll get the worst of both worlds: faster drafts and brittle decisions.

Third, learn from software teams and build feedback loops. You don’t need to turn every office into an engineering department. But you can borrow the logic: define what “good” looks like (quality criteria), test on realistic cases, review outputs, iterate, and don’t ship the tool into the wild without a way to detect when it’s going wrong. The broader point is that successful AI use tends to be less about “capability” and more about building a practice ecology around the tool.

So maybe the way forward is not to await the AGI horizon as if it were a meteorological event, like a storm front approaching that inevitably will pour down on us, but to stay closer to the ground. On Christmas, the apps I built were satisfying partly because they felt like craft. They were small, situated, and immediately answerable to my own workflow. They did not demand that I believe in a mythic future. They just made a few things easier and left the rest of the day intact. That seems like a good scale for our ambitions, at least for now. We should build practices that keep humans in the loop not as ceremonial overseers, but as responsible participants. We should design organisations that can live with probabilistic tools without pretending they are deterministic machines and make room (time, actually) for the slow work of checking, caring, and thinking together.

If we can do that, generative AI might become less of a prophecy and more of what tools have always been at their best, which is a way of supporting life without displacing it.

AI Disclosure:

Generative AI was used in the following ways for this text: ChatGPT 5.2 was utilized for experimenting with early drafts and ideas for the text. The author used the model to iterate on and test different structures for the text and to iterate on paragraphs. Nano Banana Pro was used to create the image accompanying the text.

For any questions/comments, kindly email digma@mdu.se