The core competency of LLMs is to generate text. This implies
that they are quite capable and useful for many tasks related to writing,
from synthesizing sentences based on bullet points, changing the style
of text, editing text, evaluating styles to generating titles, headlines,
and tweets. I cover each in turn in the following.
Basic writing tasks do not generally require the most cutting-edge
LLM available....For example, to synthesize basic text,
any of the models listed in Table will do
a good job. However, when the LLM is asked to go beyond the immediate
instructions, differences start to emerge based on the capabilities
of each model. At the time of writing, tools for synthesizing text
are also starting to be included in office packages such as MS Office
and Google Workspace as well as the associated email programs Outlook
and Gmail. Alternatively, a dedicated writing assistant that offers
plugins for many common applications (including web browsers and office
packages) and that suggests corrections and stylistic improvements
as the user is typing is grammarly.com.
Synthesizing text
One of the most useful capabilities of LLMs for writing assistance
may be to translate rough bullet points into well-structured sentences
that are clear and easy to read. The following is an example. Note
that the bullets also include requests for the LLM to provide examples
and offer solutions.
The advent of large language models (LLMs) that automate a plethora
of cognitive tasks presents a significant paradigm shift that academic
economists should be cognizant of, given the cognitive nature of our
work, which includes tasks such as data analysis, economic forecasting,
and policy recommendation. The rapid advancements in language models
could potentially disrupt our profession, necessitating a proactive
approach to this emerging technology. In the short term, these models
can augment our productivity by automating routine tasks, thereby
allowing us to focus on more complex and nuanced aspects of economic
research. However, in the long run, there is a substantial likelihood
that these models could displace human economists, as they become
increasingly adept at performing complex cognitive tasks. Consequently,
it is incumbent upon us to prepare for this eventuality. This could
involve developing a deeper understanding of these models to leverage
their capabilities effectively, diversifying our skill sets to include
areas not easily automated, and advocating for policies that ensure
a just transition for those potentially displaced by these technological
advancements.
Utilizing this allows researchers to concentrate their
energy on the ideas in their text as opposed to the mundane writing
process. The text is generally high-quality, although some claim that
it is a bit sterile and may lack the idiosyncrasies and elements of
surprise that characterize human writing.*This is a feature that detectors of LLM-written text zero in on, for
example the tool GPTZero, which is available at https://gptzero.me/.
However, it should be noted that such tools are not always reliable
and can easily be circumvented.
LLMs are very responsive to requests to write in different
styles. I obtained both impressive and hilarious results when I replaced
"in the style of an academic economist'' with ...the
request to write:
in a colloquial style
in a style that is comprehensible to non-economists
in the style of [enter your favorite economist, politician, etc.]*When I requested "in the style of a gangster,'' the LLM started
with "Listen here, see, these big shot large language models (LLMs)
are muscling in on our turf...''
LLMs can also write text in LaTeX format, which automatically
surrounds variables by the feX, and it can
draft beamer slides in LaTeX code based on bullet points.
Writing is of course a skill that is not only useful for research
- cognitive workers in many areas report using, e.g., ChatGPT to draft
emails, memos, performance reviews, or summaries based on a few bullet
points. Likewise, professors report generating the first draft of
letters of recommendation or syllabi using LLMs.
Editing text (for mistakes, style, clarity, simplicity, ... )
Another useful capability is editing. LLMs can edit text for
grammatical or spelling mistakes, style, clarity, or simplicity. This
set of capabilities is useful for all writers, but perhaps most for
non-native speakers who want to improve their writing. In the following
example, I have bolded each correction made by the LLM for emphasis:
This paper describes six domains in which
large language models (LLMs) display emergent capabilities
that are useful for economic research.
Sure, here are the main corrections:
1. "These paper" was corrected to "This paper". The subject-verb agreement was incorrect. If the subject is singular (like "paper"), the verb should also be singular (like "describes").
2. "describe" was corrected to "describes". The verb should match the singular subject "paper".
3. "domain" was corrected to "domains". The context implies that there are multiple domains, so the plural form should be used.
4. "displays" was corrected to "display". The subject "LLMs" is plural, so the verb should also be plural.
5. "emergant" was corrected to "emergent". This was a spelling mistake.
6. "to the economic research" was corrected to "for economic research". The preposition "for" is more appropriate in this context.
As illustrated in the follow-up question, the LLM can
also largely explain its edits so that writers can learn from any
corrections it has made. This latter capability has improved significantly
going from GPT-3.5 to GPT-4. Notice, however, that it left out a seventh
correction from the list — it also corrected a typo in the word
"capabilities.'' This highlights the need to carefully check the
results of LLMs.
Another useful LLM application is to convert text to be comprehensible
to readers of different levels, for example, using a prompt like "Rewrite
the following text so that it is comprehensible to an eight-year-old'',
"comprehensible to an eighth-grader,'' or "comprehensible to
a policymaker without economics education.''
Evaluating text
LLMs can also evaluate text for its style, its clarity, or
similar. The following question about a draft version of the abstract
of this paper turned out to be a bit of a deflating experience:
1. Lack of Clarity: The abstract uses jargon and technical terms without
providing clear definitions or explanations, making it difficult for
readers unfamiliar with the topic to understand.
2. Overuse of Passive Voice: The abstract frequently uses passive
voice, which can make the writing seem less direct and engaging.
3. Lack of Specificity: The abstract mentions " dozens
of use cases" and " six domains"
but does not provide any specific examples or details.
4. Redundancy: The abstract repeats the idea of AI's potential benefits
for economists multiple times without adding new information.
5. Lack of Structure: The abstract does not have a clear structure
or flow, making it difficult for readers to follow the author's argument.
Generative AI, specifically large language models (LLMs) like ChatGPT, holds the potential to transform research. This paper explores the application of LLMs in six key areas: ideation and feedback, writing, background research, data analysis, coding, and mathematical derivations. For each area, I provide practical instructions and real-world examples, categorizing the usefulness of LLMs from experimental to highly beneficial.
The focus of this paper is on the potential benefits for economists. By leveraging generative AI, economists can automate micro tasks, leading to significant productivity gains. As AI systems continue to evolve, these benefits are expected to increase.
The paper also delves into the long-term implications of AI-powered cognitive automation for economic research. Accompanying online resources provide a step-by-step guide on how to get started with generative AI, along with regular updates on its latest capabilities beneficial for economists.
The shortcomings identified by the LLM are well-taken,
and the rewritten version does a decent job of alleviating some but
not all of them.
Other useful LLM prompts for style evaluation ...include:
Which arguments are the hardest to follow in the following text?
Evaluate the clarity and coherence of the following text.
Are there any ambiguities in the following text?
Generating titles & headlines
Another useful application is to come up with catchy headlines
or paper titles, as illustrated by the following example based on
the version of the abstract used in the previous example:
1. " Revolutionizing Economic Research: The Power of
Generative AI and Large Language Models"
2. " Unleashing AI in Economics: A Comprehensive Guide
to Utilizing Large Language Models"
3. " The Future of Economic Research: Harnessing the
Potential of Generative AI and Large Language Models"
1. "Generative AI in Economic Research"
2. "Large Language Models: A New Tool for Economists"
3. "The Role of AI in Modern Economic Studies"
The suggested titles fit the described work quite well
and adhere closely to the requested styles.
Generating tweets to promote a paper
After having written a paper, the next step is to promote
it. The following exercise is similar to the previous one but converts
the abstract I pasted into chunks that would work well for Twitter
or similar social networks:
1. " Generative AI, especially large language models
like ChatGPT, are set to revolutionize research! They're becoming
handy as research assistants and tutors across six domains. #AIRevolution
#ChatGPT"
2. " From ideation to coding, generative AI is proving
useful in a variety of tasks. Economists, it's time to automate your
micro tasks and boost productivity! #AIProductivity #Economics"
3. " The future of AI-powered cognitive automation in
economic research looks promising. Stay tuned for regular updates
on the latest capabilities of generative AI. #FutureOfAI #EconomicResearch"
4. " Want to get started with generative AI? Check out
the online resources associated with my paper for instructions and
updates. #GenerativeAI #AIResources"
Upon request, LLMs can also incorporate suitable emoticons
or links.