builder: release-mozilla-release-fennec_bouncer_submitter slave: bld-linux64-spot-539 starttime: 1455059145.5 results: success (0) buildid: 20160209150441 builduid: 9a071d8113e54283bd11b5b77ec25949 ========= Started set props: master (results: 0, elapsed: 0 secs) (at 2016-02-09 15:05:45.498322) ========= master: http://buildbot-master73.bb.releng.usw2.mozilla.com:8001/ ========= Finished set props: master (results: 0, elapsed: 0 secs) (at 2016-02-09 15:05:45.498770) ========= ========= Started set props: basedir (results: 0, elapsed: 0 secs) (at 2016-02-09 15:05:45.499061) ========= bash -c pwd in dir /builds/slave/rel-m-rel-bncr_sub-00000000000/. (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', 'pwd'] environment: BUILDBOTCUSTOM=https://hg.mozilla.org/build/buildbotcustom BUILDBOT_CONFIGS=https://hg.mozilla.org/build/buildbot-configs CCACHE_HASHDIR= CLOBBERER_URL=https://api.pub.build.mozilla.org/clobberer/forceclobber G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-539.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-m-rel-bncr_sub-00000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False /builds/slave/rel-m-rel-bncr_sub-00000000000 program finished with exit code 0 elapsedTime=0.004248 basedir: '/builds/slave/rel-m-rel-bncr_sub-00000000000' ========= master_lag: 0.05 ========= ========= Finished set props: basedir (results: 0, elapsed: 0 secs) (at 2016-02-09 15:05:45.555732) ========= ========= Started downloading to buildprops.json (results: 0, elapsed: 0 secs) (at 2016-02-09 15:05:45.556038) ========= ========= Finished downloading to buildprops.json (results: 0, elapsed: 0 secs) (at 2016-02-09 15:05:45.567363) ========= ========= Started 'rm -rf ...' (results: 0, elapsed: 0 secs) (at 2016-02-09 15:05:45.567647) ========= rm -rf properties in dir /builds/slave/rel-m-rel-bncr_sub-00000000000/. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-rf', 'properties'] environment: BUILDBOTCUSTOM=https://hg.mozilla.org/build/buildbotcustom BUILDBOT_CONFIGS=https://hg.mozilla.org/build/buildbot-configs CCACHE_HASHDIR= CLOBBERER_URL=https://api.pub.build.mozilla.org/clobberer/forceclobber G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-539.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-m-rel-bncr_sub-00000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.002601 ========= master_lag: 0.04 ========= ========= Finished 'rm -rf ...' (results: 0, elapsed: 0 secs) (at 2016-02-09 15:05:45.613698) ========= ========= Started set props: script_repo_url (results: 0, elapsed: 0 secs) (at 2016-02-09 15:05:45.613968) ========= script_repo_url: https://hg.mozilla.org/build/mozharness ========= Finished set props: script_repo_url (results: 0, elapsed: 0 secs) (at 2016-02-09 15:05:45.614287) ========= ========= Started 'bash -c ...' (results: 0, elapsed: 0 secs) (at 2016-02-09 15:05:45.614588) ========= bash -c 'wget -Oarchiver_client.py --no-check-certificate --tries=10 --waitretry=3 https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py' in dir /builds/slave/rel-m-rel-bncr_sub-00000000000/. (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', 'wget -Oarchiver_client.py --no-check-certificate --tries=10 --waitretry=3 https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py'] environment: BUILDBOTCUSTOM=https://hg.mozilla.org/build/buildbotcustom BUILDBOT_CONFIGS=https://hg.mozilla.org/build/buildbot-configs CCACHE_HASHDIR= CLOBBERER_URL=https://api.pub.build.mozilla.org/clobberer/forceclobber G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-539.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-m-rel-bncr_sub-00000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False --2016-02-09 15:05:45-- https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py Resolving hg.mozilla.org... 63.245.215.102, 63.245.215.25 Connecting to hg.mozilla.org|63.245.215.102|:443... connected. HTTP request sent, awaiting response... 200 Script output follows Length: 12141 (12K) [text/x-python] Saving to: ‘archiver_client.py’ 0K .......... . 100% 150M=0s 2016-02-09 15:05:45 (150 MB/s) - ‘archiver_client.py’ saved [12141/12141] program finished with exit code 0 elapsedTime=0.297818 ========= master_lag: 0.01 ========= ========= Finished 'bash -c ...' (results: 0, elapsed: 0 secs) (at 2016-02-09 15:05:45.923839) ========= ========= Started 'rm -rf ...' (results: 0, elapsed: 0 secs) (at 2016-02-09 15:05:45.924148) ========= rm -rf scripts in dir /builds/slave/rel-m-rel-bncr_sub-00000000000/. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-rf', 'scripts'] environment: BUILDBOTCUSTOM=https://hg.mozilla.org/build/buildbotcustom BUILDBOT_CONFIGS=https://hg.mozilla.org/build/buildbot-configs CCACHE_HASHDIR= CLOBBERER_URL=https://api.pub.build.mozilla.org/clobberer/forceclobber G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-539.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-m-rel-bncr_sub-00000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.003236 ========= master_lag: 0.04 ========= ========= Finished 'rm -rf ...' (results: 0, elapsed: 0 secs) (at 2016-02-09 15:05:45.971693) ========= ========= Started 'bash -c ...' (results: 0, elapsed: 2 mins, 21 secs) (at 2016-02-09 15:05:45.971973) ========= bash -c 'python archiver_client.py mozharness --repo releases/mozilla-release --tag FENNEC_44_0_2_RELEASE --destination scripts --debug' in dir /builds/slave/rel-m-rel-bncr_sub-00000000000/. (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', 'python archiver_client.py mozharness --repo releases/mozilla-release --tag FENNEC_44_0_2_RELEASE --destination scripts --debug'] environment: BUILDBOTCUSTOM=https://hg.mozilla.org/build/buildbotcustom BUILDBOT_CONFIGS=https://hg.mozilla.org/build/buildbot-configs CCACHE_HASHDIR= CLOBBERER_URL=https://api.pub.build.mozilla.org/clobberer/forceclobber G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-539.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-m-rel-bncr_sub-00000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False 2016-02-09 15:05:46,018 Setting DEBUG logging. 2016-02-09 15:05:46,018 "FENNEC_44_0_2_RELEASE" was passed as the tag. Querying remote repository for corresponding rev hash. 2016-02-09 15:05:46,664 revision being used: 0f5e5278a10a 2016-02-09 15:05:46,664 attempt 1/10 2016-02-09 15:05:46,664 Getting archive location from https://api.pub.build.mozilla.org/archiver/hgmo/releases/mozilla-release/0f5e5278a10a?&preferred_region=us-west-2&suffix=tar.gz&subdir=testing/mozharness 2016-02-09 15:05:46,883 attempt 1/10 2016-02-09 15:05:47,218 current task status: no status available at this point. state: PENDING 2016-02-09 15:05:47,218 sleeping for 11.00s (attempt 1/10) 2016-02-09 15:05:58,230 attempt 2/10 2016-02-09 15:05:58,578 current task status: no status available at this point. state: PENDING 2016-02-09 15:05:58,579 sleeping for 15.50s (attempt 2/10) 2016-02-09 15:06:14,094 attempt 3/10 2016-02-09 15:06:14,467 current task status: no status available at this point. state: PENDING 2016-02-09 15:06:14,467 sleeping for 23.25s (attempt 3/10) 2016-02-09 15:06:37,741 attempt 4/10 2016-02-09 15:06:38,084 current task status: no status available at this point. state: PENDING 2016-02-09 15:06:38,084 sleeping for 35.88s (attempt 4/10) 2016-02-09 15:07:13,995 attempt 5/10 2016-02-09 15:07:14,340 current task status: Task has expired from pending for too long. Re-creating task. state: RETRY 2016-02-09 15:07:14,340 sleeping for 52.81s (attempt 5/10) 2016-02-09 15:08:07,206 attempt 6/10 2016-02-09 15:08:07,384 current task status: Task completed! Check 's3_urls' for upload locations. state: SUCCESS 2016-02-09 15:08:07,448 unpacking tar archive at: mozilla-release-0f5e5278a10a/testing/mozharness/ program finished with exit code 0 elapsedTime=141.591574 ========= master_lag: 0.02 ========= ========= Finished 'bash -c ...' (results: 0, elapsed: 2 mins, 21 secs) (at 2016-02-09 15:08:07.582679) ========= ========= Started set props: script_repo_revision (results: 0, elapsed: 0 secs) (at 2016-02-09 15:08:07.583001) ========= echo FENNEC_44_0_2_RELEASE in dir /builds/slave/rel-m-rel-bncr_sub-00000000000/. (timeout 1200 secs) watching logfiles {} argv: ['echo', 'FENNEC_44_0_2_RELEASE'] environment: BUILDBOTCUSTOM=https://hg.mozilla.org/build/buildbotcustom BUILDBOT_CONFIGS=https://hg.mozilla.org/build/buildbot-configs CCACHE_HASHDIR= CLOBBERER_URL=https://api.pub.build.mozilla.org/clobberer/forceclobber G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-539.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-m-rel-bncr_sub-00000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False FENNEC_44_0_2_RELEASE program finished with exit code 0 elapsedTime=0.005487 script_repo_revision: 'FENNEC_44_0_2_RELEASE' ========= master_lag: 0.05 ========= ========= Finished set props: script_repo_revision (results: 0, elapsed: 0 secs) (at 2016-02-09 15:08:07.636242) ========= ========= Started downloading to oauth.txt (results: 0, elapsed: 0 secs) (at 2016-02-09 15:08:07.636557) ========= ========= Finished downloading to oauth.txt (results: 0, elapsed: 0 secs) (at 2016-02-09 15:08:07.653389) ========= ========= Started 'scripts/scripts/bouncer_submitter.py -c ...' (results: 0, elapsed: 0 secs) (at 2016-02-09 15:08:07.653697) ========= scripts/scripts/bouncer_submitter.py -c releases/bouncer_fennec.py --revision FENNEC_44_0_2_RELEASE --repo releases/mozilla-release --version 44.0.2 --credentials-file oauth.txt --bouncer-api-prefix https://bounceradmin.mozilla.com/api --build-number 2 in dir /builds/slave/rel-m-rel-bncr_sub-00000000000/. (timeout 1200 secs) watching logfiles {} argv: ['scripts/scripts/bouncer_submitter.py', '-c', 'releases/bouncer_fennec.py', '--revision', 'FENNEC_44_0_2_RELEASE', '--repo', 'releases/mozilla-release', '--version', '44.0.2', '--credentials-file', 'oauth.txt', '--bouncer-api-prefix', 'https://bounceradmin.mozilla.com/api', '--build-number', '2'] environment: BUILDBOTCUSTOM=https://hg.mozilla.org/build/buildbotcustom BUILDBOT_CONFIGS=https://hg.mozilla.org/build/buildbot-configs CCACHE_HASHDIR= CLOBBERER_URL=https://api.pub.build.mozilla.org/clobberer/forceclobber G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-539.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PROPERTIES_FILE=/builds/slave/rel-m-rel-bncr_sub-00000000000/buildprops.json PWD=/builds/slave/rel-m-rel-bncr_sub-00000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False 15:08:07 INFO - MultiFileLogger online at 20160209 15:08:07 in /builds/slave/rel-m-rel-bncr_sub-00000000000 15:08:07 INFO - Run as scripts/scripts/bouncer_submitter.py -c releases/bouncer_fennec.py --revision FENNEC_44_0_2_RELEASE --repo releases/mozilla-release --version 44.0.2 --credentials-file oauth.txt --bouncer-api-prefix https://bounceradmin.mozilla.com/api --build-number 2 15:08:07 INFO - Dumping config to /builds/slave/rel-m-rel-bncr_sub-00000000000/logs/localconfig.json. 15:08:07 INFO - {'append_to_log': False, 15:08:07 INFO - 'base_work_dir': '/builds/slave/rel-m-rel-bncr_sub-00000000000', 15:08:07 INFO - 'bouncer-api-prefix': 'https://bounceradmin.mozilla.com/api', 15:08:07 INFO - 'build_number': '2', 15:08:07 INFO - 'config_files': ('releases/bouncer_fennec.py',), 15:08:07 INFO - 'credentials_file': 'oauth.txt', 15:08:07 INFO - 'log_level': 'info', 15:08:07 INFO - 'log_to_console': True, 15:08:07 INFO - 'opt_config_files': (), 15:08:07 INFO - 'products': {'apk': {'add-locales': False, 15:08:07 INFO - 'paths': {'android-api-11': {'bouncer-platform': 'android', 15:08:07 INFO - 'path': '/mobile/releases/%(version)s/android-api-11/:lang/fennec-%(version)s.:lang.android-arm.apk'}, 15:08:07 INFO - 'android-api-9': {'bouncer-platform': 'android-api-9', 15:08:07 INFO - 'path': '/mobile/releases/%(version)s/android-api-9/:lang/fennec-%(version)s.:lang.android-arm.apk'}, 15:08:07 INFO - 'android-x86': {'bouncer-platform': 'android-x86', 15:08:07 INFO - 'path': '/mobile/releases/%(version)s/android-x86/:lang/fennec-%(version)s.:lang.android-i386.apk'}}, 15:08:07 INFO - 'product-name': 'Fennec-%(version)s', 15:08:07 INFO - 'ssl-only': False}}, 15:08:07 INFO - 'repo': 'releases/mozilla-release', 15:08:07 INFO - 'revision': 'FENNEC_44_0_2_RELEASE', 15:08:07 INFO - 'version': '44.0.2', 15:08:07 INFO - 'volatile_config': {'actions': None, 'add_actions': None, 'no_actions': None}, 15:08:07 INFO - 'work_dir': 'build'} 15:08:07 INFO - ##### 15:08:07 INFO - ##### Running clobber step. 15:08:07 INFO - ##### 15:08:07 INFO - Running main action method: clobber 15:08:07 INFO - rmtree: /builds/slave/rel-m-rel-bncr_sub-00000000000/build 15:08:07 INFO - ##### 15:08:07 INFO - ##### Running download-shipped-locales step. 15:08:07 INFO - ##### 15:08:07 INFO - Running main action method: download_shipped_locales 15:08:07 INFO - No need to download shipped-locales 15:08:07 INFO - ##### 15:08:07 INFO - ##### Running submit step. 15:08:07 INFO - ##### 15:08:07 INFO - Running main action method: submit 15:08:07 INFO - Checking if Fennec-44.0.2 already exists 15:08:07 INFO - retry: Calling _api_call with args: ('product_show?product=Fennec-44.0.2', None), kwargs: {}, attempt #1 15:08:07 INFO - Submitting to https://bounceradmin.mozilla.com/api/product_show?product=Fennec-44.0.2 15:08:08 INFO - Server response 15:08:08 INFO - 15:08:08 INFO - Products found: 1 15:08:08 WARNING - Product Fennec-44.0.2 already exists. Skipping... 15:08:08 INFO - Copying logs to upload dir... 15:08:08 INFO - mkdir: /builds/slave/rel-m-rel-bncr_sub-00000000000/build/upload/logs program finished with exit code 0 elapsedTime=0.537798 ========= master_lag: 0.02 ========= ========= Finished 'scripts/scripts/bouncer_submitter.py -c ...' (results: 0, elapsed: 0 secs) (at 2016-02-09 15:08:08.210937) ========= ========= Started no change (results: 0, elapsed: 0 secs) (at 2016-02-09 15:08:08.215338) ========= bash -c 'for file in `ls -1`; do cat $file; done' in dir /builds/slave/rel-m-rel-bncr_sub-00000000000/properties (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', 'for file in `ls -1`; do cat $file; done'] environment: BUILDBOTCUSTOM=https://hg.mozilla.org/build/buildbotcustom BUILDBOT_CONFIGS=https://hg.mozilla.org/build/buildbot-configs CCACHE_HASHDIR= CLOBBERER_URL=https://api.pub.build.mozilla.org/clobberer/forceclobber G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-539.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-m-rel-bncr_sub-00000000000/properties SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.005317 ========= master_lag: 0.07 ========= ========= Finished no change (results: 0, elapsed: 0 secs) (at 2016-02-09 15:08:08.295646) ========= ========= Started 'rm -f ...' (results: 0, elapsed: 0 secs) (at 2016-02-09 15:08:08.295990) ========= rm -f oauth.txt in dir /builds/slave/rel-m-rel-bncr_sub-00000000000/. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-f', 'oauth.txt'] environment: BUILDBOTCUSTOM=https://hg.mozilla.org/build/buildbotcustom BUILDBOT_CONFIGS=https://hg.mozilla.org/build/buildbot-configs CCACHE_HASHDIR= CLOBBERER_URL=https://api.pub.build.mozilla.org/clobberer/forceclobber G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-539.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-m-rel-bncr_sub-00000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.003451 ========= master_lag: 0.12 ========= ========= Finished 'rm -f ...' (results: 0, elapsed: 0 secs) (at 2016-02-09 15:08:08.420885) ========= ========= Total master_lag: 0.43 =========