AI Is Rewiring How Knowledge Is Created—and That’s Becoming a Problem

Introduction

Every technological revolution has changed how humans work.

The printing press changed how knowledge spread.

The internet changed how information was accessed.

Artificial intelligence is changing something even more fundamental—how knowledge itself is created.

From drafting research papers and reviewing manuscripts to tutoring students and analysing data, AI is beginning to participate in almost every stage of the knowledge creation process. Tasks that once took days or weeks—literature reviews, statistical analysis, coding, proofreading and drafting—can now be completed in a fraction of the time.

Yet the same technology that accelerates knowledge creation is quietly changing something more fundamental. Increasingly, humans are not simply delegating routine work to AI—they are delegating parts of the thinking process itself.

The question is no longer whether AI can help produce knowledge. It is whether knowledge created with less human judgement remains equally trustworthy.

The Explosion of Knowledge Production

Since the release of ChatGPT in 2022, in the space of a few prompts, researchers can now summarise hundreds of papers, generate literature reviews, draft manuscripts and analyse datasets that once required weeks of work. Unsurprisingly, the barriers to producing academic work have fallen dramatically. One recent study found that submissions to an academic journal increased by 42 per cent after generative AI became widely available. Yet that surge came alongside a measurable decline in writing quality. This should not be surprising. When producing a publishable first draft becomes significantly easier, the incentive shifts from developing better ideas to producing more papers.

Every major technological breakthrough reduces the cost of producing something valuable. The printing press reduced the cost of books. Digital cameras reduced the cost of photography. The internet reduced the cost of publishing. Generative AI is now reducing the cost of producing intellectual work. When the cost of production falls, output almost always rises—and academic research is proving no exception. Tasks that previously required teams of graduate research assistants—summarising literature, cleaning data, drafting reports and proofreading manuscripts—can increasingly be completed by a single researcher working alongside AI.

Similar patterns have emerged across every industry transformed by digital technology. Blogging dramatically increased the number of published opinions. YouTube lowered the barriers to video creation, leading to an explosion of content. Social media made publishing instantaneous, but also flooded users with low-quality information. AI appears to be triggering the same economic dynamic in academic research: more output, lower production costs and increasing pressure to distinguish genuine insight from automated content.

When Quantity Begins to Undermine Quality

A surge in research output does not automatically translate into better science. In fact, when the barriers to producing research fall, the quality of that research can become increasingly difficult to maintain. Medicine offers one of the clearest examples of this emerging problem.

A study led by Joshua Wang at Taipei’s Tzu Chi Hospital examined research papers produced by medical students using AI-assisted analysis of healthcare records. The researchers found recurring methodological weaknesses that suggested AI was making it easier to produce studies without fully understanding the statistical methods behind them. “Ease of access to big data and automated analysis tools can facilitate the rapid generation of poorly designed epidemiological studies, which collectively pose a risk to the quality of medical literature,”. The finding is significant because it suggests the problem is not AI itself but the ease with which sophisticated analytical tools can now be used without a corresponding understanding of research methodology.

One of the most common mistakes in data analysis is confusing correlation with causation. Two variables may appear to move together without one actually causing the other. AI can rapidly identify statistical relationships, but it cannot determine whether those relationships make scientific sense. Scientific significance depends not only on mathematics but also on biological plausibility, study design and existing evidence—areas where human expertise remains indispensable. That judgement still depends on the researcher.The risks include meaningless correlations which are not causal. 

The same study also warned about ‘P-hacking’—the practice of repeatedly analysing data until a statistically significant result appears. AI makes this process dramatically easier because it can test hundreds or even thousands of possible relationships in minutes. While this increases the chances of finding an impressive-looking result, it also increases the likelihood of publishing findings that are simply the product of chance rather than genuine scientific discovery.

Traditionally, analysing a medical dataset required months of training in statistics, study design and epidemiology. The effort involved meant that researchers developed an understanding of why particular statistical methods were appropriate and where their limitations lay. Today, generative AI can perform many of the same calculations within minutes. The technical barrier has fallen dramatically, but the intellectual responsibility has not. Understanding the results remains just as important as generating them.Consider a simple example. An AI System identifies ten statistically significant relationships within a dataset. At first glance, this appears impressive. Yet statistical significance alone does not establish scientific importance. Some of those relationships may be coincidental, others may be driven by hidden variables, and some may disappear entirely when tested on new data. Without careful human judgement, researchers risk mistaking mathematical patterns for meaningful scientific discoveries.

The problem, therefore, is not that AI performs statistical analysis. Statistical software has assisted researchers for decades. What generative AI changes is the ease with which complex analyses can be produced—and the temptation to accept those analyses without understanding how they were derived. Science has always depended on scepticism as much as discovery. Every result must be questioned, challenged and replicated before it becomes accepted knowledge. AI can accelerate the discovery of potential patterns, but it cannot replace the intellectual discipline required to decide whether those patterns deserve to be believed. As the cost of producing research continues to fall, the value of human judgement will only become more important.

The Collapse Of Peer Review

If AI is making it easier to produce research, then the next question becomes equally important: who ensures that this growing volume of research still meets scientific standards? The answer has traditionally been peer review—the process through which independent experts scrutinise a study’s methods, evidence and conclusions before publication. Yet AI is beginning to reshape this process as well.

Luca Fraccascia, an associate professor at Sapienza University of Rome, recently posted that when he shared a draft paper with a reviewer for a journal that he helps edit, he received a 2,127-word reply within 14 minutes. “Such timing is, to say the least, unusual and raises concerns regarding the integrity of the review process,” he said.

A detailed review normally requires reading the manuscript carefully, checking the methodology, evaluating the evidence and considering whether the conclusions follow logically from the data. Completing all of that within fourteen minutes naturally raises questions about whether the review reflected genuine scholarly evaluation or was substantially assisted by AI.
Why this raises question regarding the integrity of the review process. Peer review is built on more than proofreading. It relies on careful reading, independent judgement, methodological expertise and constructive criticism. These are precisely the aspects of intellectual work that generative AI can imitate convincingly, creating the temptation to delegate parts of the review process rather than engage fully with the manuscript.

There is also a deeper tension between the purpose of AI and the purpose of peer review. Generative AI is designed to produce fluent, plausible and helpful responses. Peer review, by contrast, is intentionally sceptical. Its purpose is not to generate convincing text but to identify weak arguments, methodological flaws and unsupported conclusions. When reviewers rely too heavily on AI, there is a risk that critique becomes less rigorous and more formulaic, weakening one of science’s most important quality-control mechanisms.

The concerns extend beyond review quality. Peer reviewers are entrusted with confidential manuscripts that often contain unpublished ideas, proprietary data and years of original work. Uploading such material into external AI systems raises difficult questions about intellectual property, confidentiality and informed consent. Even where AI providers promise strong privacy protections, the act of sharing unpublished research with third-party systems introduces new risks that traditional peer review was never designed to address.

AI may help summarise a manuscript or identify missing references, but the responsibility for evaluating evidence, questioning assumptions and exercising scholarly judgement cannot be delegated. Scientific publishing depends not only on producing knowledge but also on ensuring that knowledge has been critically examined by another human expert. 

If research papers are the products of science, then peer review is its quality-control department. Automating parts of that process may improve efficiency, but if the inspection itself becomes superficial, the entire production system becomes less reliable.

AI can help write reviews, but it cannot replace the scepticism that gives peer review its value.

AI Doesn’t Think—It Predicts

Behind every discussion about AI in research and education lies a more fundamental question: what exactly is AI doing when it appears to think? The answer is critical because much of the current debate assumes that AI reasons in the same way humans do. It does not.

Generative AI does not understand ideas in the way people do. It analyses enormous amounts of text and predicts which sequence of words is most likely to follow a given prompt. The result is often remarkably fluent, coherent and useful. Yet fluency should not be mistaken for understanding. AI produces answers that are statistically probable, not conclusions that it knows to be true.

Imagine asking an AI system, “Is this hypothesis correct?” The model cannot independently verify the hypothesis through reasoning or experimentation. Instead, it generates a response that resembles what a well-written answer to that question would typically look like based on patterns in the data it has seen. That response may be insightful, partially correct or entirely wrong—but the model itself has no awareness of which is true.

This distinction explains one of AI’s greatest strengths and one of its greatest weaknesses. Because it is optimised to generate fluent language, AI often expresses uncertainty with the same confidence that it expresses established facts. The quality of its writing can therefore create an illusion of reliability, encouraging users to trust answers that still require independent verification.

Human reasoning involves questioning assumptions, weighing competing explanations, recognising uncertainty and deciding when the available evidence is insufficient. These are not simply language tasks; they are acts of judgement. AI can imitate the language associated with those processes, but it cannot independently exercise them.

Because AI can generate persuasive language at almost zero marginal cost, the economic incentive increasingly shifts towards producing more intellectual output. The challenge for researchers, educators and institutions is ensuring that the quality of human judgement keeps pace with the speed of machine generation.

AI generates language; humans generate judgement.

Education Is Facing the Same Problem

The same shift that is transforming academic research is now reshaping education. As AI becomes an everyday learning companion, students are increasingly delegating not only routine tasks but also parts of the thinking process itself. The result is a growing debate over whether AI is enhancing learning—or quietly replacing the mental effort through which learning occurs.

A growing body of research suggests that students are increasingly using AI not simply as a reference tool but as a substitute for developing critical thinking and independent judgement. Tasks that once required analysing evidence, constructing arguments or solving problems can now be completed with a few carefully written prompts.

Recognising this risk, many schools and universities continue to rely on high-stakes, invigilated examinations without access to digital devices. These assessments are designed to measure what students understand independently rather than what they can generate with technological assistance.

Research has shown that while AI can improve short-term performance and increase the speed with which students complete tasks, those benefits may disappear once the technology is removed. In some studies, students who relied heavily on AI later performed worse than peers who had completed the work independently.

The finding suggests that AI may sometimes improve performance without improving understanding.

Researchers describe this phenomenon as cognitive offloading—transferring mental tasks to external tools instead of performing them ourselves. Offloading routine activities such as checking grammar or organising notes can be beneficial because it frees mental capacity for more demanding work. Problems arise when higher-order thinking—evaluating evidence, constructing arguments or solving unfamiliar problems—is also delegated.

The Australian Network for Quality Digital Education warns that excessive reliance on AI can create a sense of “false mastery.” Students may feel confident because they can produce polished assignments with AI’s assistance, even though they have not developed the underlying understanding themselves. Over time, this can contribute to what researchers describe as “cognitive atrophy”—the gradual weakening of skills that are no longer regularly exercised.

History offers a useful comparison. When calculators became widely available, students stopped performing lengthy arithmetic calculations by hand. Few educators considered this a problem because arithmetic was never the ultimate goal of mathematics; mathematical reasoning was. AI presents a different challenge. It is no longer automating calculation alone—it is beginning to automate explanation, argument construction and problem-solving. Those activities lie much closer to the heart of learning itself.

Education has never been solely about producing correct answers. Its deeper purpose is to cultivate curiosity, judgement and the ability to reason independently. AI can support that process, but it cannot replace it. When students begin outsourcing thinking rather than routine work, the efficiency gained today may come at the cost of intellectual capability tomorrow.

None of this suggests that AI has no place in education. Used thoughtfully, it can provide personalised explanations, instant feedback and opportunities for practice that were previously unavailable to many learners. The challenge lies not in whether students use AI, but in deciding which parts of learning should remain fundamentally human.

The Productivity Paradox

Despite the risks discussed so far, it would be a mistake to conclude that AI is making research and education worse. In many respects, the opposite is true. For experienced researchers, AI is dramatically increasing productivity, allowing them to tackle questions that would previously have required far more time, money and personnel.

Igor Strebulaev, a professor at Stanford University’s Graduate School of Business, argues that AI has transformed the way he works. Tasks such as collecting data, organising information and performing preliminary analyses—once requiring lengthy projects involving multiple research assistants—can now be completed far more quickly. Rather than reducing the scope of his work, the technology has enabled him to pursue more ambitious research questions.

Interestingly, Strebulaev does not believe AI reduces the need for researchers. Instead, he argues that increased efficiency has encouraged him to undertake more projects. As he puts it, he has “never been busier” because AI allows him to work “better and deeper.” Productivity has expanded the frontier of what is possible rather than simply reducing workload.

This illustrates the central paradox of AI. The same technology that allows experts to accomplish more can also encourage novices to think less. Experienced researchers often use AI to automate repetitive tasks while retaining responsibility for judgement and interpretation. Students, by contrast, may be tempted to delegate the judgement itself before they have developed it.

Experience changes the way AI is used. Experts possess the knowledge needed to evaluate AI-generated outputs, recognise mistakes and challenge weak conclusions. Beginners often lack that foundation. As a result, the same tool that amplifies the capabilities of an experienced researcher may inadvertently replace the learning process for a student.

Economically, AI behaves like many earlier general-purpose technologies. By reducing the time required for routine intellectual tasks, it increases the productivity of skilled workers. History suggests that such technologies rarely eliminate work altogether. Instead, they often enable people to undertake more complex and ambitious work than was previously feasible.

The introduction of spreadsheets did not eliminate accountants; it allowed them to analyse larger businesses and more complex financial problems. Similarly, AI is unlikely to eliminate researchers. Instead, it may redefine which parts of research require uniquely human expertise.

Education, however, presents a different challenge. Universities are not merely trying to increase productivity; they are trying to develop capability. A technology that helps experts work faster may not necessarily help beginners become experts.

The lesson is not that AI benefits researchers while harming students. Rather, AI rewards those who already possess strong judgement and risks weakening those who are still developing it. In other words, AI amplifies existing capability—it does not automatically create it.

AI makes experts more productive; it does not automatically make beginners more capable.

The Real Question Isn’t Whether AI Is Good

After examining how AI is reshaping research, peer review and education, it becomes clear that the central debate is not whether AI is good or bad. That framing is too simplistic. The more important question is which parts of intellectual work should be delegated to machines and which should remain fundamentally human.

Throughout history, major technologies have reshaped work rather than simply eliminating it. They have rewarded those who learned to use new tools effectively while reducing the value of skills that could be performed more efficiently by machines.

Consider the spreadsheet. Before applications such as Excel became widespread, accountants and financial analysts spent countless hours performing manual calculations. Excel did not eliminate the need for financial expertise; it enabled professionals to analyse larger datasets, model more complex scenarios and make better-informed decisions. The routine calculations were automated, but judgement remained a human responsibility.

Calculators produced a similar shift. They automated arithmetic, allowing students and professionals to spend more time on mathematical reasoning rather than repetitive computation. The educational goal was never to eliminate thinking, but to redirect it towards higher-order problems.

Generative AI represents the next stage of this historical pattern. The difference is that it does not merely automate calculations or data entry. It increasingly automates language, analysis and even the appearance of reasoning. That makes deciding what should remain a human responsibility far more important than with previous technologies.

The greatest benefits are realised when AI automates routine intellectual tasks—summarising documents, organising information, checking grammar, drafting initial outlines or identifying patterns in data—while humans retain responsibility for judgement, creativity and critical thinking. Problems emerge when that balance is reversed and AI is asked to make decisions, evaluate evidence or construct arguments in place of the user.

In those situations, AI no longer functions as an assistant to human intelligence; it begins to substitute for it. Over time, the risk is not simply lower-quality work, but the gradual erosion of the skills that individuals need to perform that work independently.

Every transformative technology changes what humans do. AI is unique because it changes how humans think. The future of education and research will therefore depend less on how intelligent AI becomes and more on whether humans continue to exercise the judgement, curiosity and scepticism that no machine can genuinely possess. AI should not become a replacement for human thinking. It should become the tool that allows human thinking to reach further.

The future belongs not to those who avoid AI, but to those who know when not to rely on it.

CONCLUSION

The evidence from research laboratories, academic journals and classrooms points towards the same conclusion. The challenge is not whether AI should be used in education and research. That question has already been answered. AI is here to stay. The real challenge is deciding which parts of intellectual work should be delegated to machines and which must remain fundamentally human.

Grammar can be automated.
Formatting can be automated.
Literature searches can be automated.
Data cleaning can be automated.
Curiosity cannot be automated.

Critical judgement cannot be automated.
Intellectual honesty cannot be automated.
Scientific scepticism cannot be automated.

The distinction is simple but profound: AI should automate execution, not judgement.

AI is at its best when it extends human capability rather than replaces human reasoning. It can help researchers analyse larger datasets, assist reviewers in identifying missing references and enable students to receive personalised explanations. But the responsibility for asking better questions, challenging assumptions and deciding what deserves to be believed must remain human.

Every generation inherits knowledge created by the one before it. Universities, scientific journals and classrooms exist not merely to distribute information but to cultivate the judgement needed to create trustworthy knowledge for the future. If AI weakens that judgement, the consequences extend far beyond individual students or researchers. They shape the quality of future scientific discovery itself.

The future of education and research will not be determined by how intelligent AI becomes. It will be determined by whether humans continue to exercise the curiosity, scepticism and judgement that AI can imitate but not genuinely possess. The greatest risk is not that machines will begin thinking like humans. It is that humans will gradually stop thinking for themselves.

Knowledge has always begun with a question. That question must remain human.