copy_tree not copying sub directories

Question

Using Python 3.8 on Windows 10 OS. The first time I run copy_tree it works perfectly, i.e. all files and sub-directories are copied over as expected.

When I manually delete the newly copied directory from the destination and run copy_tree again (within 1 minute from deleting the initial destination directory created by copy_tree), it will copy everything but the sub-directories. Note that there is no content within these sub-folders.

Is there a timeframe where a deleted folder is not technically deleted?

EDIT: Here are my functions:

def isFolder(path):
    return os.path.isdir(path) if os.path.exists(path) else False

def copyFolderContent(srcFolder, destFolder):
    srcFolderOK = isFolder(srcFolder)
    destFolderOK = isFolder(destFolder)

    if srcFolderOK and destFolderOK:
        return copy_tree(srcFolder, destFolder)
    else:
        return None

The source directory looks like this:

SourceParentFolder
   -> ChildFolder

The destination directory looks like this:

DestinationParentFolder
   (no contents)

After running copy_tree the destination directory is:

DestinationParentFolder
   -> ChildFolder

Then, after manually deleting "ChildFolder" and running copy_tree within approximately one minute:

DestinationParentFolder
   (no contents)

The expectation is that "ChildFolder" is copied into DestinationParentFolder, but this does not happen unless a significant amount of time has passed.

Answer 1

I am having a very similar, perhaps identical, problem. After some testing, it seems the issue might partly be with my IDE, Spyder.

The Issue

I have two directories /test/a and /test/x. I want to copy the contents of a into x. I want preserve as much metadata as possible, and I don't want my script to fail for existing files (as was the case in this question), so I'm trying to use distutils.dir_util.copy_tree().

a contains the following files and subdirectories:

+---a
    |   
    \---b
        |   testfile1.txt
        |   
        \---c
                testfile2.txt
                testfile3.txt

Yesterday, I created a new anaconda environment with Python 3.9.2 and Spyder 4.2.3.

I ran the following script in Spyder:

from pathlib import Path
from distutils.dir_util import copy_tree

src = Path.cwd() / 'a'
dst = Path.cwd() / 'x'


copy_tree(str(src), str(dst), preserve_times=True, update=False)

The first time I run the script, all of a's contents are copied into x.

If I delete x and re-run the script, a new x is created, the contents of a are not copied, and the following errors are thrown:

Traceback (most recent call last):

  File "C:\Users\Joseph\computer\Anaconda3\envs\sandbox3\lib\distutils\file_util.py", line 41, in _copy_file_contents
    fdst = open(dst, 'wb')

FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\Joseph\\test\\x\\b\\c\\testfile2.txt'


During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "C:\Users\Joseph\test\troubleshoot-copy_tree-0.2.py", line 14, in <module>
    copy_tree(str(src), str(dst), preserve_times=True, update=False)

  File "C:\Users\Joseph\computer\Anaconda3\envs\sandbox3\lib\distutils\dir_util.py", line 157, in copy_tree
    copy_tree(src_name, dst_name, preserve_mode,

  File "C:\Users\Joseph\computer\Anaconda3\envs\sandbox3\lib\distutils\dir_util.py", line 157, in copy_tree
    copy_tree(src_name, dst_name, preserve_mode,

  File "C:\Users\Joseph\computer\Anaconda3\envs\sandbox3\lib\distutils\dir_util.py", line 161, in copy_tree
    copy_file(src_name, dst_name, preserve_mode,

  File "C:\Users\Joseph\computer\Anaconda3\envs\sandbox3\lib\distutils\file_util.py", line 151, in copy_file
    _copy_file_contents(src, dst)

  File "C:\Users\Joseph\computer\Anaconda3\envs\sandbox3\lib\distutils\file_util.py", line 43, in _copy_file_contents
    raise DistutilsFileError(

DistutilsFileError: could not create 'C:\Users\Joseph\test\x\b\c\testfile2.txt': No such file or directory

The error seems to be caused by _copy_file_contents, which is trying to open and write a new file where none exists. This doesn't make sense to me, because there shouldn't be a file there in the first place. See this unanswered question for a very similar problem, but with shutil.copytree() instead.

What I tried

I tried deleting x again, then exiting out of Spyder, starting a new anaconda terminal session, opening Spyder again, and re-running the script. Everything works as expected (x is created, and a's contents are copied into it). If I subsequently delete x and re-run the script, x is created and then the same errors are thrown. I don't know why restarting the Anaconda prompt and Spyder solves the problem for the first script run, but I was able to reproduce this multiple times.

I tried moving a and my script file into a new parent directory, and the script worked again (created x, copied all of a, etc). In the new location, I then deleted x and re-ran the script. x was created and then the same errors were thrown.

I then deleted x, exited out of Spyder, and started IDLE from the Anaconda prompt. I ran the script in IDLE and it worked the first time. I deleted x and re-ran the script, and it worked again. I did this several more times and it worked each time.

Why is my script working in IDLE but not in Spyder?

My apologies for my ridiculously long answer/question. I didn't want to post a separate question, as this sounded like a duplicate. Hopefully, this issue will get some more attention.

System Info: Windows 10, Python 3.9.2, Spyder 4.2.3