LLMs are quite capable at coding tasks, especially in the
languages python and R, for which the most online resources are available.
LLMs can write, edit, modify, translate, or debug snippets of code
based on instructions in plain English (or other natural languages).
Moreover, they can act as tutors when using new libraries, functions,
or even programming languages that the user is not very familiar with
...by quickly generating output that shows what libraries
and functions are needed for specific types of operations or what
syntactic structures to use in a given programming language. This
allows the human programmer to consult the LLM and weave together
code from many different snippets generated by it.
The reasons why LLMs are so proficient at coding include the following:
There are vast repositories of code available online that are included
in their training data, for example from GitHub. The syntax of computer
code seems to be relatively easy to learn for these systems. Moreover,
the AI labs producing cutting-edge LLMs themselves benefit from the
code generation abilities of these systems, which provides them with
strong incentives for improving these capabilities. In fact, coding
may be one of the areas where current LLMs lead to the greatest productivity
gains: Peng et al. (2023)Peng, S., Kalliamvakou, E., Cihon, P., and Demirer, M. (2023). The impact of AI on developer productivity: Evidence from GitHub Copilot. report a controlled experiment in which programmers
using OpenAI-powered GitHub Copilot completed their assignments on
average 55.8% faster, amounting to a 126% productivity increase.
In the OpenAI ecosystem, Advanced Data Analysis has made the
use of ChatGPT Plus for coding tasks in the programming language python
even more convenient, as it can write code, execute it, learn from
it, write follow-up code, and display the requested results. ...It
also allows users to upload code, ask for specific modifications,
and download it again to continue work on it. GitHub Copilot offers
seamless integration into Visual Studio and several other integrated
development environments for programmers, and works for many different
programming and markup languages.*While writing this paper, I used ChatGPT Plus to write LaTeX code.
I asked it how to define the boxes around the sample prompts throughout
the paper in a way that they are visually appealing and break across
pages, and I asked it to produce Table in
LaTeX based on the table of contents for section 3. I used GitHub
Copilot when working on the replication code for this section. Meta's Code LlaMA, released in August 2023, offers powerful coding
assistance on an open-source basis.
Writing code
Alright, let's simulate the Solow growth model. To simplify things, let's consider the Solow-Swan model without technological progress. The model describes the evolution of capital (K), labor (L), and output (Y) in an economy. The basic equation for the change in capital stock over time is given by:
\\fracdKdt=sY - \\delta K
where:
s is the savings rate
Y is the total output, which is given by a production function, usually a Cobb-Douglas type: Y=K^\\alpha L^1-\\alpha,
\\delta is the depreciation rate.
Assumptions for the model:
Labor grows at a constant rate n, i.e., L(t) = L(0)e^nt,
The savings rate s, the depreciation rate \\delta, and the labor growth rate n are all constants.
To get started, we need to specify the values of these parameters. For example, we can use the following:
s = 0.4,
\\delta = 0.1,
n = 0.02,
\\alpha = 0.33,
initial capital stock K(0) = 1,
initial labor L(0)=1.
To simulate the model, we'll use the Euler method for integrating the differential equations. This method approximates the value of the function at a future time as the value at the current time plus the derivative of the function at the current time multiplied by the time step.
Then, we will plot the following four charts:
Capital over time,
Labor over time,
Output over time,
Capital per worker over time.
Does that sound good?
I just provided a short "Go ahead'' to continue the generation
of the program:
%
import numpy as np
import matplotlib.pyplot as plt
# Model parameters
s = 0.4
delta = 0.1 n = 0.02 alpha = 0.33 K0 = 1 L0 = 1 T = 100 # Total simulation time
# Time grid dt = 0.01 times = np.arange(0, T, dt)
# Initialize arrays to store the values of K, L, Y over time
K = np.zeros(len(times)) L = np.zeros(len(times)) Y = np.zeros(len(times))
# Plot capital over time
axs[0, 0].plot(times, K)
axs[0, 0].set_title('Capital over time')
axs[0, 0].set_xlabel('Time')
axs[0, 0].set_ylabel('Capital')
# Plot labor over time
axs[0, 1].plot(times, L)
axs[0, 1].set_title('Labor over time')
axs[0, 1].set_xlabel('Time')
axs[0, 1].set_ylabel('Labor')
# Plot output over time
axs[1, 0].plot(times, Y)
axs[1, 0].set_title('Output over time')
axs[1, 0].set_xlabel('Time')
axs[1, 0].set_ylabel('Output')
# Plot capital per worker over time
axs[1, 1].plot(times, K_L)
axs[1, 1].set_title('Capital per worker over time')
axs[1, 1].set_xlabel('Time')
axs[1, 1].set_ylabel('Capital per worker')
# Make the layout tight
plt.tight_layout()
plt.show()
Here are the four plots for the Solow growth model:
Capital over time: The capital stock increases over time and approaches a steady state. This steady state is the point at which investment (saving) equals depreciation, so the capital stock no longer changes. (Author's note: although the numerical simulation is correct, this point of the description is clearly wrong)
Labor over time: Labor grows exponentially at a constant rate, reflecting the assumption of constant population growth in the Solow model.
Output over time: Output also increases over time and approaches a steady state. This is because output is a function of capital and labor, which are both increasing. However, due to diminishing returns to capital, the growth of output slows down over time.
Capital per worker over time: Capital per worker also increases over time and approaches a steady state. This is the point at which the economy achieves balanced growth in the Solow model. At the steady state, output per worker and capital per worker grow at the same rate, which is the rate of technological progress. In this simplified version of the model without technological progress, this growth rate is zero, so output per worker and capital per worker are constant at the steady state.
These results illustrate the key predictions of the Solow growth model: in the long run, an economy approaches a steady state level of capital (both total and per worker), and this steady state level is determined by the savings rate, the depreciation rate, and the rate of population growth. Changes in these parameters can lead to a higher or lower steady state, but they cannot sustain long-run economic growth. According to the model, long-run economic growth can only be sustained by technological progress.
The model performed the simulation task as requested.
It started with a detailed description of the Solow model, introduced
parameter assumptions, asked for user approval, and wrote and executed
the correct python code. However, it also volunteered to interpret
the results and incorrectly described the path of capital over time
as "approaching a steady state.'' Since it assumed positive population
growth, this is clearly wrong and also contradicts the simulation
displayed in the top left panel of the figure. (Advanced Data Analysis
can generate figures but cannot actually "see'' them so it simply
hallucinated the description.) Moreover, in a concluding paragraph,
the LLM wrote: "These results illustrate the key predictions of
the Solow growth model: in the long run, an economy approaches a steady
state level of capital (both total and per worker),'' which — under
positive population growth — is clearly impossible.
I also attempted to simulate the Ramsey model. The LLM correctly
generated code to simulate the path of consumption and capital for
given initial conditions, but it did not manage to solve for the correct
starting point for consumption, ...even after I added
a reminder to my prompt that initial consumption needs to lie on the
stable arm of a saddle-path stable system.
This is representative of the state of LLM coding capabilities
at the time of writing — LLMs are useful assistants to automate
many of the mundane parts of programming, but they require human oversight
for the more difficult parts. Moreover, LLMs can also act as powerful
tutors that help users to find the packages and functions that they
need for specific use cases.
Explaining code
Whereas the previous example wrote code based on English language
instructions, LLMs can also do the reverse and explain what a given
piece of code does in plain English, akin to a tutor. The following
examples demonstrates this use case:
This function performs a two-step generalized
least squares regression.
In the first step, it performs an ordinary least squares regression
to estimate the parameters (beta1) of the model.
In the second step, it uses the residuals from the first step to estimate
the variance of the errors, and then performs a weighted least squares
regression to estimate the parameters (beta2) of the model, taking
into account the heteroscedasticity (non-constant variance) of the
errors.
The function returns the estimated parameters from both steps.
This functionality is particularly useful when using sources that
have not documented their code well.
Translating code
LLMs can also convert code in one programming language into another
language, for example to port a project to another platform or to
integrate code snippets found in an online coding forum such as StackExchange
that is in the wrong language.
The following prompt, building on the preceding conversation, provides
an example:
Sure, here is the equivalent MATLAB code for the function:
function [beta1, beta2] = regression(X, y) X = [ones(size(X, 1), 1), X]; XtXinv = inv(X' * X); beta1 = XtXinv * X' * y; residuals = y - X * beta1; V = diag(residuals.\^2); Vinv = inv(V); XtVinvXinv = inv(X' * (Vinv * X)); beta2 = XtVinvXinv * X' * (Vinv * y);
end
Please note that in MATLAB, the operator for matrix multiplication is `*`, not `np.dot()`. Also, the operator for element-wise power is `.\^`, not `**`.
Current LLMs are quite reliable in translating short
pieces of code across common programming languages. For longer sequences,
human assistance is still required.
Debugging code
One of the most frustrating parts of programming is to debug code.
LLMs also exhibit useful capabilities in this domain. They can not
only help to interpret error messages but also verify code.
In my first example, I simply pasted a typical error message
from python that is obtained when the user forgets to import a library,
without any further instructions:
The error message is telling you that the name 'np' is not defined. This usually happens when you try to use a library that you haven't imported yet.
In your case, 'np' is a common alias for the numpy library. You can fix the error by importing numpy at the beginning of your script. Here's how you do it:
"`python import numpy as np "`
After adding this line, your script will know what 'np' is and it should work as expected.
The LLM correctly interpreted my prompt as a request
for help to debug the error. It recognized the source of the error,
explained it, and provided the code needed to rectify it.
The next example, building on the python function two examples
earlier but with an error introduced, illustrates how LLMs can help
to catch mistakes such as typos in code:
The function seems to be implementing a form of weighted least squares
regression, but there are a few issues with it:
1. The function uses numpy functions and methods but numpy is not
imported. You need to import numpy at the beginning of your script.
2. The variable `Vinv` is not defined. It seems like it should be
`V_inv`.
Here is the corrected function: [Generated correct code]
The first point observes that the code does not run unless
the numpy package is imported (as in the previous example); the second
point correctly caught the spelling mistake that I introduced.
More generally, LLMs are also sometimes able to detect when
indices are mixed up, and to provide advice when one describes the
ways in which a program malfunctions. They are less capable at detecting
high-level mistakes in the algorithms underlying code — for this,
human debugging is usually still required.