Create a "black box" test case
This transcript shows aider creating a black box test case, without being given access to the source code of the function being tested or any of the other code in the repo.
Instead, aider is operating entirely off a high level map of the repository based on ctags, which provides details on all the symbols and function signatures in the repo. Using only the meta-data in the map, the LLM is able to figure out how to call the method to be tested, as well as how to instantiate multiple class objects that are required to prepare for the test.
In detail, aider is able to use the repo map to:
Find the function signature of the
cmd_add()
function which the user wants a test case for.Determine that it is a method of the
Command
class, so the test case will need to instantiate an instance to conduct the test.Identify that creating a
Command
instance requires passing inInputOutput
andCoder
instances.Figure out the arguments required to instantiate the
InputOutput
instance.Decide that the
Coder
class looks complex enough to use aMagickMock
.
The cmd_add()
method is defined without type annotations, so aider makes a reasonable but incorrect guess that it accepts a list
of files. The user invokes the /run
command to run the new unit test, which throws an error based on this misunderstanding.
The LLM reviews the error message and fixes the call to cmd_add()
to correctly pass a str
with a space-separated list of filenames instead. The test then passes on the next /run
.
$ aider tests/test_commands.py Creating empty file tests/test_commands.py Added tests/test_commands.py to the chat Files not tracked in .git:
tests/test_commands.py
Add them? y Added tests/test_commands.py to the git repo Commit b9cea2c Added new files to the git repo: tests/test_commands.py Using git repo: .git
add a test for cmd_add that passes in 'foo.txt' and 'bar.txt' and ensures they have both been created after the test. run the test in a tmp dir.
I will add a new test method called test_cmd_add
in the tests/test_commands.py
file. This test will create a temporary directory, change the current working directory to the temporary directory, and then call the cmd_add
method with 'foo.txt' and 'bar.txt' as arguments. After that, it will check if both files have been created in the temporary directory.
Here's the code change:
This test will ensure that the cmd_add
method creates the specified files in the current working directory.
Applied edit to tests/test_commands.py Commit e2d5212 aider: Added a test method
test_cmd_add
that creates a temporary directory, calls thecmd_add
method with 'foo.txt' and 'bar.txt', and checks if both files have been created in the temporary directory.
#### /run pytest tests/test_commands.py
Add the output to the chat? [y/n] (y): y
It seems that the cmd_add
method in aider/commands.py
is expecting a string as input, but in the test, we are passing a list. I will update the test to pass a string instead of a list.
Here's the code change:
Now, run the test again with the updated test method:
Applied edit to tests/test_commands.py Commit d1551b4 aider: Updated test to pass a string instead of a list when calling
cmd_add
method.
#### /run pytest tests/test_commands.py
Last updated