builder: mozilla-beta_win8_64_test-other-e10s-pgo slave: t-w864-ix-088 starttime: 1493022928.65 results: failure (2) buildid: 20170423234919 builduid: 0fd5de7718bf4e8b9d2a272010b6d3e9 revision: 3c4807a601c88f1f9aca07dc20ebdbd3cebbf01c ========= Started set props: master (results: 0, elapsed: 0 secs) (at 2017-04-24 01:35:28.647567) ========= master: http://buildbot-master127.bb.releng.scl3.mozilla.com:8201/ ========= Finished set props: master (results: 0, elapsed: 0 secs) (at 2017-04-24 01:35:28.648021) ========= ========= Started downloading to buildprops.json (results: 0, elapsed: 0 secs) (at 2017-04-24 01:35:28.648300) ========= ========= Finished downloading to buildprops.json (results: 0, elapsed: 0 secs) (at 2017-04-24 01:35:28.861260) ========= ========= Started 'bash -c ...' (results: 0, elapsed: 1 secs) (at 2017-04-24 01:35:28.861681) ========= 'bash' '-c' 'wget -Oarchiver_client.py --no-check-certificate --tries=10 --waitretry=3 https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py' in dir C:\slave\test\. (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', 'wget -Oarchiver_client.py --no-check-certificate --tries=10 --waitretry=3 https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py'] environment: ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\cltbld.T-W864-IX-088\AppData\Roaming COMMONPROGRAMFILES=C:\Program Files (x86)\Common Files COMMONPROGRAMFILES(X86)=C:\Program Files (x86)\Common Files COMMONPROGRAMW6432=C:\Program Files\Common Files COMPUTERNAME=T-W864-IX-088 COMSPEC=C:\windows\system32\cmd.exe DCLOCATION=SCL3 DNSSUFFIX=wintest.releng.scl3.mozilla.com FP_NO_HOST_CHECK=NO HOMEDRIVE=C: HOMEPATH=\Users\cltbld.T-W864-IX-088 KTS_HOME=C:\Program Files\KTS KTS_VERSION=1.19c LOCALAPPDATA=C:\Users\cltbld.T-W864-IX-088\AppData\Local LOGONSERVER=\\T-W864-IX-088 MONDIR=C:\Monitor_config\ MOZBUILDDIR=C:\mozilla-build\ MOZ_CRASHREPORTER_NO_REPORT=1 MOZ_NO_REMOTE=1 NO_EM_RESTART=1 NUMBER_OF_PROCESSORS=8 OS=Windows_NT OURDRIVE=C: PATH=C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\mozilla-build\python27;C:\mozilla-build\python27\Scripts;C:\mozilla-build\msys\bin;C:\mozilla-build\vim\vim72;C:\mozilla-build\wget;C:\mozilla-build\info-zip;C:\CoreUtils\bin;C:\mozilla-build\buildbotve\scripts;C:\mozilla-build\hg PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=x86 PROCESSOR_ARCHITEW6432=AMD64 PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 30 Stepping 5, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=1e05 PROGRAMDATA=C:\ProgramData PROGRAMFILES=C:\Program Files (x86) PROGRAMFILES(X86)=C:\Program Files (x86) PROGRAMW6432=C:\Program Files PROMPT=$P$G PSMODULEPATH=C:\windows\system32\WindowsPowerShell\v1.0\Modules\ PUBLIC=C:\Users\Public PWD=C:\slave\test RUNLOGFILE=C:\slave\\runslave.log SLAVEDIR=C:\slave\ SYSTEMDRIVE=C: SYSTEMROOT=C:\windows TEMP=C:\Users\CLTBLD~1.T-W\AppData\Local\Temp TEST1=testie TMP=C:\Users\CLTBLD~1.T-W\AppData\Local\Temp USERDOMAIN=T-W864-IX-088 USERDOMAIN_ROAMINGPROFILE=T-W864-IX-088 USERNAME=cltbld USERPROFILE=C:\Users\cltbld.T-W864-IX-088 WINDIR=C:\windows XPCOM_DEBUG_BREAK=warn using PTY: False --01:35:29-- https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py => `archiver_client.py' Resolving hg.mozilla.org... 63.245.215.25, 63.245.215.102 Connecting to hg.mozilla.org|63.245.215.25|:443... connected. WARNING: Certificate verification error for hg.mozilla.org: certificate signature failure HTTP request sent, awaiting response... 200 Script output follows Length: 12,179 (12K) [text/x-python] 0K .......... . 100% 11.17 MB/s 01:35:29 (11.17 MB/s) - `archiver_client.py' saved [12179/12179] program finished with exit code 0 elapsedTime=1.291000 ========= master_lag: 0.03 ========= ========= Finished 'bash -c ...' (results: 0, elapsed: 1 secs) (at 2017-04-24 01:35:30.184073) ========= ========= Started 'rm -rf ...' (results: 0, elapsed: 3 secs) (at 2017-04-24 01:35:30.184470) ========= 'rm' '-rf' 'scripts' 'properties' in dir C:\slave\test\. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-rf', 'scripts', 'properties'] environment: ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\cltbld.T-W864-IX-088\AppData\Roaming COMMONPROGRAMFILES=C:\Program Files (x86)\Common Files COMMONPROGRAMFILES(X86)=C:\Program Files (x86)\Common Files COMMONPROGRAMW6432=C:\Program Files\Common Files COMPUTERNAME=T-W864-IX-088 COMSPEC=C:\windows\system32\cmd.exe DCLOCATION=SCL3 DNSSUFFIX=wintest.releng.scl3.mozilla.com FP_NO_HOST_CHECK=NO HOMEDRIVE=C: HOMEPATH=\Users\cltbld.T-W864-IX-088 KTS_HOME=C:\Program Files\KTS KTS_VERSION=1.19c LOCALAPPDATA=C:\Users\cltbld.T-W864-IX-088\AppData\Local LOGONSERVER=\\T-W864-IX-088 MONDIR=C:\Monitor_config\ MOZBUILDDIR=C:\mozilla-build\ MOZ_CRASHREPORTER_NO_REPORT=1 MOZ_NO_REMOTE=1 NO_EM_RESTART=1 NUMBER_OF_PROCESSORS=8 OS=Windows_NT OURDRIVE=C: PATH=C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\mozilla-build\python27;C:\mozilla-build\python27\Scripts;C:\mozilla-build\msys\bin;C:\mozilla-build\vim\vim72;C:\mozilla-build\wget;C:\mozilla-build\info-zip;C:\CoreUtils\bin;C:\mozilla-build\buildbotve\scripts;C:\mozilla-build\hg PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=x86 PROCESSOR_ARCHITEW6432=AMD64 PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 30 Stepping 5, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=1e05 PROGRAMDATA=C:\ProgramData PROGRAMFILES=C:\Program Files (x86) PROGRAMFILES(X86)=C:\Program Files (x86) PROGRAMW6432=C:\Program Files PROMPT=$P$G PSMODULEPATH=C:\windows\system32\WindowsPowerShell\v1.0\Modules\ PUBLIC=C:\Users\Public PWD=C:\slave\test RUNLOGFILE=C:\slave\\runslave.log SLAVEDIR=C:\slave\ SYSTEMDRIVE=C: SYSTEMROOT=C:\windows TEMP=C:\Users\CLTBLD~1.T-W\AppData\Local\Temp TEST1=testie TMP=C:\Users\CLTBLD~1.T-W\AppData\Local\Temp USERDOMAIN=T-W864-IX-088 USERDOMAIN_ROAMINGPROFILE=T-W864-IX-088 USERNAME=cltbld USERPROFILE=C:\Users\cltbld.T-W864-IX-088 WINDIR=C:\windows XPCOM_DEBUG_BREAK=warn using PTY: False program finished with exit code 0 elapsedTime=3.414000 ========= master_lag: 0.02 ========= ========= Finished 'rm -rf ...' (results: 0, elapsed: 3 secs) (at 2017-04-24 01:35:33.622641) ========= ========= Started 'bash -c ...' (results: 0, elapsed: 2 secs) (at 2017-04-24 01:35:33.623032) ========= 'bash' '-c' u'python archiver_client.py mozharness --repo releases/mozilla-beta --rev 3c4807a601c88f1f9aca07dc20ebdbd3cebbf01c --destination scripts --debug' in dir C:\slave\test\. (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', u'python archiver_client.py mozharness --repo releases/mozilla-beta --rev 3c4807a601c88f1f9aca07dc20ebdbd3cebbf01c --destination scripts --debug'] environment: ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\cltbld.T-W864-IX-088\AppData\Roaming COMMONPROGRAMFILES=C:\Program Files (x86)\Common Files COMMONPROGRAMFILES(X86)=C:\Program Files (x86)\Common Files COMMONPROGRAMW6432=C:\Program Files\Common Files COMPUTERNAME=T-W864-IX-088 COMSPEC=C:\windows\system32\cmd.exe DCLOCATION=SCL3 DNSSUFFIX=wintest.releng.scl3.mozilla.com FP_NO_HOST_CHECK=NO HOMEDRIVE=C: HOMEPATH=\Users\cltbld.T-W864-IX-088 KTS_HOME=C:\Program Files\KTS KTS_VERSION=1.19c LOCALAPPDATA=C:\Users\cltbld.T-W864-IX-088\AppData\Local LOGONSERVER=\\T-W864-IX-088 MONDIR=C:\Monitor_config\ MOZBUILDDIR=C:\mozilla-build\ MOZ_CRASHREPORTER_NO_REPORT=1 MOZ_NO_REMOTE=1 NO_EM_RESTART=1 NUMBER_OF_PROCESSORS=8 OS=Windows_NT OURDRIVE=C: PATH=C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\mozilla-build\python27;C:\mozilla-build\python27\Scripts;C:\mozilla-build\msys\bin;C:\mozilla-build\vim\vim72;C:\mozilla-build\wget;C:\mozilla-build\info-zip;C:\CoreUtils\bin;C:\mozilla-build\buildbotve\scripts;C:\mozilla-build\hg PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=x86 PROCESSOR_ARCHITEW6432=AMD64 PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 30 Stepping 5, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=1e05 PROGRAMDATA=C:\ProgramData PROGRAMFILES=C:\Program Files (x86) PROGRAMFILES(X86)=C:\Program Files (x86) PROGRAMW6432=C:\Program Files PROMPT=$P$G PSMODULEPATH=C:\windows\system32\WindowsPowerShell\v1.0\Modules\ PUBLIC=C:\Users\Public PWD=C:\slave\test RUNLOGFILE=C:\slave\\runslave.log SLAVEDIR=C:\slave\ SYSTEMDRIVE=C: SYSTEMROOT=C:\windows TEMP=C:\Users\CLTBLD~1.T-W\AppData\Local\Temp TEST1=testie TMP=C:\Users\CLTBLD~1.T-W\AppData\Local\Temp USERDOMAIN=T-W864-IX-088 USERDOMAIN_ROAMINGPROFILE=T-W864-IX-088 USERNAME=cltbld USERPROFILE=C:\Users\cltbld.T-W864-IX-088 WINDIR=C:\windows XPCOM_DEBUG_BREAK=warn using PTY: False 2017-04-24 01:35:33,867 truncating revision to first 12 chars 2017-04-24 01:35:33,868 Setting DEBUG logging. 2017-04-24 01:35:33,868 attempt 1/10 2017-04-24 01:35:33,868 Getting archive location from https://api.pub.build.mozilla.org/archiver/hgmo/releases/mozilla-beta/3c4807a601c8?&preferred_region=us-west-2&suffix=tar.gz&subdir=testing/mozharness 2017-04-24 01:35:35,196 unpacking tar archive at: mozilla-beta-3c4807a601c8/testing/mozharness/ 2017-04-24 01:35:35,398 skipping symlink on windows: configs/single_locale/linux32.py program finished with exit code 0 elapsedTime=2.180000 ========= master_lag: 0.11 ========= ========= Finished 'bash -c ...' (results: 0, elapsed: 2 secs) (at 2017-04-24 01:35:35.914000) ========= ========= Started set props: script_repo_revision (results: 0, elapsed: 0 secs) (at 2017-04-24 01:35:35.914322) ========= script_repo_revision: 3c4807a601c88f1f9aca07dc20ebdbd3cebbf01c ========= Finished set props: script_repo_revision (results: 0, elapsed: 0 secs) (at 2017-04-24 01:35:35.914774) ========= ========= Started downloading to oauth.txt (results: 0, elapsed: 0 secs) (at 2017-04-24 01:35:35.915069) ========= ========= Finished downloading to oauth.txt (results: 0, elapsed: 0 secs) (at 2017-04-24 01:35:35.939735) ========= ========= Started 'c:/mozilla-build/python27/python -u ...' failed (results: 2, elapsed: 12 mins, 1 secs) (at 2017-04-24 01:35:35.940098) ========= 'c:/mozilla-build/python27/python' '-u' 'scripts/scripts/talos_script.py' '--suite' 'other-e10s' '--add-option' '--webServer,localhost' '--branch-name' 'Mozilla-Beta' '--cfg' 'talos/windows_config.py' '--download-symbols' 'ondemand' '--use-talos-json' '--blob-upload-branch' 'Mozilla-Beta' in dir C:\slave\test\. (timeout 3600 secs) (maxTime 7200 secs) watching logfiles {} argv: ['c:/mozilla-build/python27/python', '-u', 'scripts/scripts/talos_script.py', '--suite', 'other-e10s', '--add-option', '--webServer,localhost', '--branch-name', 'Mozilla-Beta', '--cfg', 'talos/windows_config.py', '--download-symbols', 'ondemand', '--use-talos-json', '--blob-upload-branch', 'Mozilla-Beta'] environment: ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\cltbld.T-W864-IX-088\AppData\Roaming COMMONPROGRAMFILES=C:\Program Files (x86)\Common Files COMMONPROGRAMFILES(X86)=C:\Program Files (x86)\Common Files COMMONPROGRAMW6432=C:\Program Files\Common Files COMPUTERNAME=T-W864-IX-088 COMSPEC=C:\windows\system32\cmd.exe DCLOCATION=SCL3 DNSSUFFIX=wintest.releng.scl3.mozilla.com FP_NO_HOST_CHECK=NO HOMEDRIVE=C: HOMEPATH=\Users\cltbld.T-W864-IX-088 KTS_HOME=C:\Program Files\KTS KTS_VERSION=1.19c LOCALAPPDATA=C:\Users\cltbld.T-W864-IX-088\AppData\Local LOGONSERVER=\\T-W864-IX-088 MONDIR=C:\Monitor_config\ MOZBUILDDIR=C:\mozilla-build\ MOZ_CRASHREPORTER_NO_REPORT=1 MOZ_NO_REMOTE=1 NO_EM_RESTART=1 NUMBER_OF_PROCESSORS=8 OS=Windows_NT OURDRIVE=C: PATH=C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\mozilla-build\python27;C:\mozilla-build\python27\Scripts;C:\mozilla-build\msys\bin;C:\mozilla-build\vim\vim72;C:\mozilla-build\wget;C:\mozilla-build\info-zip;C:\CoreUtils\bin;C:\mozilla-build\buildbotve\scripts;C:\mozilla-build\hg PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=x86 PROCESSOR_ARCHITEW6432=AMD64 PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 30 Stepping 5, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=1e05 PROGRAMDATA=C:\ProgramData PROGRAMFILES=C:\Program Files (x86) PROGRAMFILES(X86)=C:\Program Files (x86) PROGRAMW6432=C:\Program Files PROMPT=$P$G PROPERTIES_FILE=C:\slave\test/buildprops.json PSMODULEPATH=C:\windows\system32\WindowsPowerShell\v1.0\Modules\ PUBLIC=C:\Users\Public PWD=C:\slave\test RUNLOGFILE=C:\slave\\runslave.log SLAVEDIR=C:\slave\ SYSTEMDRIVE=C: SYSTEMROOT=C:\windows TEMP=C:\Users\CLTBLD~1.T-W\AppData\Local\Temp TEST1=testie TMP=C:\Users\CLTBLD~1.T-W\AppData\Local\Temp USERDOMAIN=T-W864-IX-088 USERDOMAIN_ROAMINGPROFILE=T-W864-IX-088 USERNAME=cltbld USERPROFILE=C:\Users\cltbld.T-W864-IX-088 WINDIR=C:\windows XPCOM_DEBUG_BREAK=warn using PTY: False 01:35:36 INFO - MultiFileLogger online at 20170424 01:35:36 in C:\slave\test 01:35:36 INFO - Run as scripts/scripts/talos_script.py --suite other-e10s --add-option --webServer,localhost --branch-name Mozilla-Beta --cfg talos/windows_config.py --download-symbols ondemand --use-talos-json --blob-upload-branch Mozilla-Beta 01:35:36 INFO - Dumping config to C:\slave\test\logs\localconfig.json. 01:35:36 INFO - {'append_to_log': False, 01:35:36 INFO - 'base_work_dir': 'C:\\slave\\test', 01:35:36 INFO - 'blob_upload_branch': 'Mozilla-Beta', 01:35:36 INFO - 'blob_uploader_auth_file': 'C:\\slave\\test\\oauth.txt', 01:35:36 INFO - 'branch': 'Mozilla-Beta', 01:35:36 INFO - 'buildbot_json_path': 'buildprops.json', 01:35:36 INFO - 'config_files': ('talos/windows_config.py',), 01:35:36 INFO - 'default_actions': ('clobber', 01:35:36 INFO - 'read-buildbot-config', 01:35:36 INFO - 'download-and-extract', 01:35:36 INFO - 'populate-webroot', 01:35:36 INFO - 'create-virtualenv', 01:35:36 INFO - 'install', 01:35:36 INFO - 'run-tests'), 01:35:36 INFO - 'default_blob_upload_servers': ('https://blobupload.elasticbeanstalk.com',), 01:35:36 INFO - 'download_minidump_stackwalk': True, 01:35:36 INFO - 'download_symbols': 'ondemand', 01:35:36 INFO - 'exes': {'easy_install': ('C:\\slave\\test\\build/venv/scripts/python', 01:35:36 INFO - 'C:\\slave\\test\\build/venv/scripts/easy_install-2.7-script.py'), 01:35:36 INFO - 'hg': 'c:/mozilla-build/hg/hg', 01:35:36 INFO - 'mozinstall': ('C:\\slave\\test\\build/venv/scripts/python', 01:35:36 INFO - 'C:\\slave\\test\\build/venv/scripts/mozinstall-script.py'), 01:35:36 INFO - 'python': 'c:/mozilla-build/python27/python.exe', 01:35:36 INFO - 'tooltool.py': ('c:/mozilla-build/python27/python.exe', 01:35:36 INFO - 'C:/mozilla-build/tooltool.py'), 01:35:36 INFO - 'virtualenv': ('c:/mozilla-build/python27/python.exe', 01:35:36 INFO - 'c:/mozilla-build/buildbotve/virtualenv.py')}, 01:35:36 INFO - 'find_links': ('http://pypi.pvt.build.mozilla.org/pub', 01:35:36 INFO - 'http://pypi.pub.build.mozilla.org/pub'), 01:35:36 INFO - 'gecko_profile': False, 01:35:36 INFO - 'gecko_profile_interval': 0, 01:35:36 INFO - 'installer_path': 'installer.exe', 01:35:36 INFO - 'log_level': 'info', 01:35:36 INFO - 'log_name': 'talos', 01:35:36 INFO - 'log_to_console': True, 01:35:36 INFO - 'metro_harness_path_frmt': '%(metro_base_path)s/metro/metrotestharness.exe', 01:35:36 INFO - 'minidump_stackwalk_path': 'win32-minidump_stackwalk.exe', 01:35:36 INFO - 'minidump_tooltool_manifest_path': 'config/tooltool-manifests/win32/releng.manifest', 01:35:36 INFO - 'opt_config_files': (), 01:35:36 INFO - 'pip_index': False, 01:35:36 INFO - 'suite': 'other-e10s', 01:35:36 INFO - 'system_bits': '32', 01:35:36 INFO - 'talos_extra_options': ('--webServer', 'localhost'), 01:35:36 INFO - 'title': 'T-W864-IX-088', 01:35:36 INFO - 'use_talos_json': True, 01:35:36 INFO - 'virtualenv_modules': ('pywin32', 'talos', 'mozinstall'), 01:35:36 INFO - 'virtualenv_path': 'C:\\slave\\test\\build/venv', 01:35:36 INFO - 'virtualenv_python_dll': 'c:/mozilla-build/python27/python27.dll', 01:35:36 INFO - 'volatile_config': {'actions': None, 'add_actions': None, 'no_actions': None}, 01:35:36 INFO - 'work_dir': 'build'} 01:35:36 INFO - [mozharness: 2017-04-24 08:35:36.503000Z] Running clobber step. 01:35:36 INFO - Running pre-action listener: _resource_record_pre_action 01:35:36 INFO - Running main action method: clobber 01:35:36 INFO - rmtree: C:\slave\test\build 01:35:36 INFO - Using _rmtree_windows ... 01:35:36 INFO - retry: Calling _rmtree_windows with args: ('C:\\slave\\test\\build',), kwargs: {}, attempt #1 01:35:36 INFO - retry: attempt #1 caught exception: (145, 'RemoveDirectory', 'The directory is not empty.') 01:35:36 INFO - retry: Failed, sleeping 60 seconds before retrying 01:36:36 INFO - retry: Calling _rmtree_windows with args: ('C:\\slave\\test\\build',), kwargs: {}, attempt #2 01:36:36 INFO - retry: attempt #2 caught exception: (145, 'RemoveDirectory', 'The directory is not empty.') 01:36:36 INFO - retry: Failed, sleeping 120 seconds before retrying 01:38:36 INFO - retry: Calling _rmtree_windows with args: ('C:\\slave\\test\\build',), kwargs: {}, attempt #3 01:38:36 INFO - retry: attempt #3 caught exception: (145, 'RemoveDirectory', 'The directory is not empty.') 01:38:36 INFO - retry: Failed, sleeping 240 seconds before retrying 01:42:36 INFO - retry: Calling _rmtree_windows with args: ('C:\\slave\\test\\build',), kwargs: {}, attempt #4 01:42:36 INFO - retry: attempt #4 caught exception: (145, 'RemoveDirectory', 'The directory is not empty.') 01:42:36 INFO - retry: Failed, sleeping 300 seconds before retrying 01:47:36 INFO - retry: Calling _rmtree_windows with args: ('C:\\slave\\test\\build',), kwargs: {}, attempt #5 01:47:36 INFO - retry: attempt #5 caught exception: (145, 'RemoveDirectory', 'The directory is not empty.') 01:47:36 FATAL - Unable to remove C:\slave\test\build! 01:47:36 FATAL - Caught exception: (145, 'RemoveDirectory', 'The directory is not empty.') 01:47:36 FATAL - Caught exception: (145, 'RemoveDirectory', 'The directory is not empty.') 01:47:36 FATAL - Caught exception: (145, 'RemoveDirectory', 'The directory is not empty.') 01:47:36 FATAL - Caught exception: (145, 'RemoveDirectory', 'The directory is not empty.') 01:47:36 FATAL - Caught exception: (145, 'RemoveDirectory', 'The directory is not empty.') 01:47:36 FATAL - Running post_fatal callback... 01:47:36 FATAL - Exiting -1 01:47:36 INFO - Running post-action listener: _resource_record_post_action 01:47:36 INFO - [mozharness: 2017-04-24 08:47:36.959000Z] Finished clobber step (failed) 01:47:36 INFO - Running post-run listener: _resource_record_post_run 01:47:36 INFO - Running post-run listener: _upload_blobber_files 01:47:36 INFO - Blob upload gear active. 01:47:36 WARNING - Blob upload directory does not exist! 01:47:36 INFO - Running post-run listener: copy_logs_to_upload_dir 01:47:36 INFO - Copying logs to upload dir... 01:47:36 INFO - Using _rmtree_windows ... 01:47:36 INFO - Using _rmtree_windows ... 01:47:36 INFO - Using _rmtree_windows ... 01:47:36 INFO - Using _rmtree_windows ... 01:47:36 INFO - Using _rmtree_windows ... 01:47:36 INFO - Using _rmtree_windows ... 01:47:36 INFO - Using _rmtree_windows ... program finished with exit code -1 elapsedTime=721.234000 ========= master_lag: 0.51 ========= ========= Finished 'c:/mozilla-build/python27/python -u ...' failed (results: 2, elapsed: 12 mins, 1 secs) (at 2017-04-24 01:47:37.683779) ========= ========= Started no change (results: 0, elapsed: 0 secs) (at 2017-04-24 01:47:37.684451) ========= 'cmd' '/C' 'for' '%f' 'in' '(*)' 'do' '@type' '%f' in dir C:\slave\test\properties (timeout 1200 secs) watching logfiles {} argv: ['cmd', '/C', 'for', '%f', 'in', '(*)', 'do', '@type', '%f'] environment: ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\cltbld.T-W864-IX-088\AppData\Roaming COMMONPROGRAMFILES=C:\Program Files (x86)\Common Files COMMONPROGRAMFILES(X86)=C:\Program Files (x86)\Common Files COMMONPROGRAMW6432=C:\Program Files\Common Files COMPUTERNAME=T-W864-IX-088 COMSPEC=C:\windows\system32\cmd.exe DCLOCATION=SCL3 DNSSUFFIX=wintest.releng.scl3.mozilla.com FP_NO_HOST_CHECK=NO HOMEDRIVE=C: HOMEPATH=\Users\cltbld.T-W864-IX-088 KTS_HOME=C:\Program Files\KTS KTS_VERSION=1.19c LOCALAPPDATA=C:\Users\cltbld.T-W864-IX-088\AppData\Local LOGONSERVER=\\T-W864-IX-088 MONDIR=C:\Monitor_config\ MOZBUILDDIR=C:\mozilla-build\ MOZ_CRASHREPORTER_NO_REPORT=1 MOZ_NO_REMOTE=1 NO_EM_RESTART=1 NUMBER_OF_PROCESSORS=8 OS=Windows_NT OURDRIVE=C: PATH=C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\mozilla-build\python27;C:\mozilla-build\python27\Scripts;C:\mozilla-build\msys\bin;C:\mozilla-build\vim\vim72;C:\mozilla-build\wget;C:\mozilla-build\info-zip;C:\CoreUtils\bin;C:\mozilla-build\buildbotve\scripts;C:\mozilla-build\hg PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=x86 PROCESSOR_ARCHITEW6432=AMD64 PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 30 Stepping 5, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=1e05 PROGRAMDATA=C:\ProgramData PROGRAMFILES=C:\Program Files (x86) PROGRAMFILES(X86)=C:\Program Files (x86) PROGRAMW6432=C:\Program Files PROMPT=$P$G PSMODULEPATH=C:\windows\system32\WindowsPowerShell\v1.0\Modules\ PUBLIC=C:\Users\Public PWD=C:\slave\test\properties RUNLOGFILE=C:\slave\\runslave.log SLAVEDIR=C:\slave\ SYSTEMDRIVE=C: SYSTEMROOT=C:\windows TEMP=C:\Users\CLTBLD~1.T-W\AppData\Local\Temp TEST1=testie TMP=C:\Users\CLTBLD~1.T-W\AppData\Local\Temp USERDOMAIN=T-W864-IX-088 USERDOMAIN_ROAMINGPROFILE=T-W864-IX-088 USERNAME=cltbld USERPROFILE=C:\Users\cltbld.T-W864-IX-088 WINDIR=C:\windows XPCOM_DEBUG_BREAK=warn using PTY: False program finished with exit code 0 elapsedTime=0.101000 ========= master_lag: 0.03 ========= ========= Finished no change (results: 0, elapsed: 0 secs) (at 2017-04-24 01:47:37.812615) ========= ========= Started 'rm -f ...' (results: 0, elapsed: 0 secs) (at 2017-04-24 01:47:37.812945) ========= 'rm' '-f' 'oauth.txt' in dir C:\slave\test\. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-f', 'oauth.txt'] environment: ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\cltbld.T-W864-IX-088\AppData\Roaming COMMONPROGRAMFILES=C:\Program Files (x86)\Common Files COMMONPROGRAMFILES(X86)=C:\Program Files (x86)\Common Files COMMONPROGRAMW6432=C:\Program Files\Common Files COMPUTERNAME=T-W864-IX-088 COMSPEC=C:\windows\system32\cmd.exe DCLOCATION=SCL3 DNSSUFFIX=wintest.releng.scl3.mozilla.com FP_NO_HOST_CHECK=NO HOMEDRIVE=C: HOMEPATH=\Users\cltbld.T-W864-IX-088 KTS_HOME=C:\Program Files\KTS KTS_VERSION=1.19c LOCALAPPDATA=C:\Users\cltbld.T-W864-IX-088\AppData\Local LOGONSERVER=\\T-W864-IX-088 MONDIR=C:\Monitor_config\ MOZBUILDDIR=C:\mozilla-build\ MOZ_CRASHREPORTER_NO_REPORT=1 MOZ_NO_REMOTE=1 NO_EM_RESTART=1 NUMBER_OF_PROCESSORS=8 OS=Windows_NT OURDRIVE=C: PATH=C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\mozilla-build\python27;C:\mozilla-build\python27\Scripts;C:\mozilla-build\msys\bin;C:\mozilla-build\vim\vim72;C:\mozilla-build\wget;C:\mozilla-build\info-zip;C:\CoreUtils\bin;C:\mozilla-build\buildbotve\scripts;C:\mozilla-build\hg PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=x86 PROCESSOR_ARCHITEW6432=AMD64 PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 30 Stepping 5, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=1e05 PROGRAMDATA=C:\ProgramData PROGRAMFILES=C:\Program Files (x86) PROGRAMFILES(X86)=C:\Program Files (x86) PROGRAMW6432=C:\Program Files PROMPT=$P$G PSMODULEPATH=C:\windows\system32\WindowsPowerShell\v1.0\Modules\ PUBLIC=C:\Users\Public PWD=C:\slave\test RUNLOGFILE=C:\slave\\runslave.log SLAVEDIR=C:\slave\ SYSTEMDRIVE=C: SYSTEMROOT=C:\windows TEMP=C:\Users\CLTBLD~1.T-W\AppData\Local\Temp TEST1=testie TMP=C:\Users\CLTBLD~1.T-W\AppData\Local\Temp USERDOMAIN=T-W864-IX-088 USERDOMAIN_ROAMINGPROFILE=T-W864-IX-088 USERNAME=cltbld USERPROFILE=C:\Users\cltbld.T-W864-IX-088 WINDIR=C:\windows XPCOM_DEBUG_BREAK=warn using PTY: False program finished with exit code 0 elapsedTime=0.101000 ========= master_lag: 0.03 ========= ========= Finished 'rm -f ...' (results: 0, elapsed: 0 secs) (at 2017-04-24 01:47:37.939189) ========= ========= Started reboot slave lost (results: 0, elapsed: 1 secs) (at 2017-04-24 01:47:37.939517) ========= ========= Finished reboot slave lost (results: 0, elapsed: 1 secs) (at 2017-04-24 01:47:39.035182) ========= ========= Total master_lag: 0.73 =========