Guest Post by Dr. Agnella Izzo Matic
TL;DR – AI is a new tool that can (and should) be leveraged to speed up some specific parts of the writing process that can be automated – saving time and mental bandwidth for the parts of the process that need idea generation and critical analysis. AI won’t do an expert job writing your manuscript or grant application for you ☹.
Everybody is talking about it. Some people are scared of it. Others have wholeheartedly embraced it and are ready to get married. We’re talking about artificial intelligence (AI), which I would argue hit mainstream awareness with the public release of ChatGPT in late 2022.
[I think machine learning (ML) is a more accurate term to describe the technology, but I will use the term “AI” here interchangeably because that’s what’s on the tip of everybody’s tongue.]
AI is more than just ChatGPT (as you’ll see below in the list of tools). ChatGPT is in the category of AI termed large language models (LLMs). If you want a deep dive on how LLMs are designed to work (and face the realization that “at the moment, we don’t have any real insight into how LLMs accomplish feats like this.”), read this article.
LLMs are trained on existing written material and generate “new” written material based on inputs and its own predictions in the model. My favorite lay description of LLMs so far is “ChatGPT doesn’t present facts. It generates fact-shaped sentences.” “Word salad” is another description that seems apt.
In my opinion, ChatGPT and other LLM-based tools could be useful for rephrasing some existing work (i.e., you want to rewrite a methods section so you’re not copy/pasting it from your old grant application or manuscript) or writing a lay summary or abstract of your work.
You may also be able to have it draft a CV/biosketch given publicly available information about you on your LinkedIn profile and your faculty webpage. LLMs may do a reasonable job at drafting a template for a cover letter or a response letter.
In all of the potential use cases, people need to add clinical relevance, strategy, and accuracy. People need to handle proprietary or confidential information. People still need to devote time to fact-checking the output of AI tools. AI can be a tool to harness (rather than be scared of) but it can’t replace your ability to generate new ideas or critically analyze and synthesize scientific research.
Here are some AI resources that may help you smooth the process of developing the grant application or improve the product:
Note 1: The AI space is evolving quickly; check to see if there are new resources available since this posting.
Note 2: Many AI tools are still being developed and updated regularly; your results with these resources may vary; play around with the resources and validate their output as needed.
- Vital – Translates any medical note into plain English (claims to be secure); https://vital.io/translate
- Elicit [paid] – Automate time-consuming research tasks like summarizing papers, extracting data, and synthesizing your findings; https://elicit.com/
- Consensus – Search engine that uses AI to find insights in research papers; https://consensus.app/search/
- System – Powered by AI and structured as a large-scale graph, System comprises millions of evidence-based relationships between tens of thousands of topics and aims to explain how everything in the world is related. https://about.system.com/
- QuillBot – initially developed for paraphrasing, also offers a grammar checker, AI writer, and summarizer; https://quillbot.com/
- ScholarAI – ScholarAI connects the LLMs that power ChatGPT with tailored access to open access peer-reviewed articles, connecting researchers with trustworthy sources and summarizing research. https://scholar-ai.net/
- ChatGPT [paid] – engaging conversations, gain insights, automate tasks; https://chat.openai.com/auth/login
- This article has great tips on how to write prompts for ChatGPT (or other LLMs) to get the most out of the outputs; https://medicalfuturist.com/prompt-engineering-11-tips-to-craft-great-chatgpt-prompts/
- You can use Faraday (https://faraday.dev/) to download a local LLM to chat with, so that none of your input or the generated chat is stored by the LLM
- Bing Chat (uses ChatGPT-4, free); AI chat companion for search engine functions; https://www.bing.com/search?q=Bing+AI&showconv=1
In an interview with Kristian Hammond, a Professor of Computer Science at Vanderbilt University, he described LLMs as using “deep learning to predict language and produce conversational text from a user prompt. They can also respond to a question with authoritative language that might have no connection to ground truth.” “…these systems were never designed to tell the truth. They were designed to be fluent. And it turns out, if you’re fluent, sometimes you tell the truth because you know how words connect to each other.”
In fact, there are examples, here and here for starters, where LLMs were tested on medical information and manuscript writing, where there were some obvious flaws in the output.
