Simple answer: GenAI tools do not always generate credible, reliable information. Your academic integrity, and your reputation as an employee, can be negatively affected by using GenAI output without careful evaluation.
The same principles for evaluating information sources apply to generative AI. Tests such as the CRAP/CRAAP and SIFT tests can be helpful in determining if the information you’ve found is reliable.
However, some of the questions we typically ask ourselves about sources may be more difficult to answer when consulting generative AI, because the process it takes to arrive at answers is not public.
Consider the following concerns as good reasons to carefully evaluate GenAI output:
As more text is published that has been created by generative AI, eventually this AI-generated content will enter the training datasets for new generations of AI. This may lead to a decrease in the quality of the data, as errors in early generations of AI may compound themselves over time.
This idea was proposed and tested by Shumailov et al. (2023) in their paper “The Curse of Recursion: Training on Generated Data Makes Models Forget.” They found that the inclusion of AI-generated content in training datasets led to what they call model collapse - "a degenerative process whereby, over time, models forget the true underlying data distribution, even in the absence of a shift in the distribution over time" (p. 2).
We're already seeing authors and creators using AI in various formats across many sites, including video and text-based content. As this becomes more wide spread, we'll need to continue to carefully evaluate content to ensure it's authenticity and accuracy.
Further reading:
Shumailov, I., Shumaylov, Z., Zhao, Y., Gal, Y., Papernot, N., & Anderson, R. (2023). The Curse of Recursion: Training on Generated Data Makes Models Forget. ArXiv, https://doi.org/10.48550/arXiv.2305.17493
The CRAP Test is a set of criteria YOU can use to determine a website's reliability, accuracy, and legitimacy. Read through the information here, and watch the attached video for a fun look at evaluating websites.
Problems viewing? Watch Evaluating Websites by Seneca (2:03 min) College Libraries on YouTube
Source: Adapted from "Evaluating Information: The CRAAP Test" by McMaster University Health Sciences Library, CC BY-NC-SA. / A derivative of the CRAAP test developed by Meriam Library, California State University, Chico, CC BY 4.0.
SIFT: a four-step technique developed by Mike Caulfield from the University of Washington, used to assess information and appropriate for anyone who engages in information-seeking behaviour.
Problems viewing? Watch SIFT The Four Moves: Evaluating Generative AI Content (4:38 min) by The Learning Portal / Le Portail d’Apprentissage on YouTube
Pause to consider the credibility of the content generated by the Artificial Intelligence before you copy or share this information.
Ask Yourself: Does this output contain relevant information for my assignment? What kind of sources were retrieved by the text generator? Is the information generated based on fact, opinion, other? Does the information contain diverse perspectives and viewpoints? How current is the information cited?
Artificial Intelligence is known to generate fictitious information. Ensure that you fact-check, or double-check the content before using it.
Ask Yourself: Do you recognize any of the authors, publishers, or websites? Are there citations? If yes, can you find the cited sources in library databases? Can you verify the accuracy of the information? Can you identify bias in the information generated by the AI, or sources represented in the text?
Locate the best evidence on your topic by searching for additional sources on library databases and search engines.
Ask Yourself: Is this the best information available on my topic? Can I locate sources that are more current than those generated by AI? Can I search for similar ideas or related topics in other peer-reviewed journals?
Instead of relying on Artificial Intelligence to provide you with the full picture, track down the source and then review the content to determine if the information is suitable for your assignment.
Ask Yourself: Can I access the books, articles, websites, and other sources cited by the AI? Is the data, facts, and details generated for me accurately represented?
Source: The SIFT Method was reused from The Learning Portal, licensed under CC BY-NC 4.0. / A derivative of SIFT (The Four Moves) by Mike Caulfield, CC BY 4.0.
Attribution from source: This content was adapted, with permission from "Evaluating GenAI Content - opens in a new tab" by Seneca Polytechnic Libraries. Along with information from "Evaluate AI Generated Content- opens in a new tab" by Sheridan Library & Learning Services and is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Evaluating AI Output is adapted from "Evaluating Information Sources" and "Generative AI and ChatGPT" by The University of British Columbia Library, licensed under CC BY-NC 4.0. / Adapted to Georgian context. Updates to headings and remixed info to focus on evaluating content. RADAR test replaced by CRAP/CRAAP test (openly licensed).
The SIFT Method was reused from The Learning Portal, licensed under CC BY-NC 4.0.
Attribution from source:, with permission from "Evaluating GenAI Content - opens in a new tab" by Seneca Polytechnic Libraries. Along with information from "Evaluate AI Generated Content- opens in a new tab" by Sheridan Library & Learning Services and is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.