Bluemix Spark: spark-submit failing when downloading stderr and stdout? -

i using spark service in ibm bluemix. trying launch java piece of code executing spark process using spark-submit.sh script.

my command line is:

./spark-submit.sh --vcap ./vcap.json --deploy-mode cluster --class org.apache.spark.examples.javasparkpi \ --master https://169.54.219.20 ~/documents/spark/javasparkpi.jar  

i using latest spark-submit.sh version (as of yesterday).

./spark-submit.sh --version spark-submit.sh  version : '1.0.0.0.20160330.1' 

this worked fine couple of weeks ago (with old spark-submit.sh) getting following error:

downloading stdout_1461024849908170118   % total    % received % xferd  average speed   time    time     time  current                                  dload  upload   total   spent    left  speed   0    89    0    89    0     0     56      0 --:--:--  0:00:01 --:--:--   108 failed download workdir/driver-20160418191414-0020-5e7fb175-6856-4980-97bc-8e8aa0d1f137/stdout     stdout_1461024849908170118  downloading stderr_1461024849908170118   % total    % received % xferd  average speed   time    time     time  current                                  dload  upload   total   spent    left  speed   0    89    0    89    0     0     50      0 --:--:--  0:00:01 --:--:--   108 failed download workdir/driver-20160418191414-0020-5e7fb175-6856-4980-97bc-8e8aa0d1f137/stderr     stderr_1461024849908170118 

any ideas on doing wrong? in advance.

edit:

by looking @ log file have found problem not while downloading stdout , stderr when submitting job.

{   "action" : "submissionstatusresponse",   "driverstate" : "failed",   "message" : "exception cluster: org.apache.spark.sparkexception: failed change container cwd org.apache.spark.deploy.master.egoapplicationmanager.egodriverexitcallback(egoapplicationmanager.scala:168) org.apache.spark.deploy.master.masterscheduledelegatordriver.oncontainerexit(masterscheduledelegatordriver.scala:144) org.apache.spark.deploy.master.resourcemanager.resourcemanageregoslot.handleactivityfinish(resourcemanageregoslot.scala:555) org.apache.spark.deploy.master.resourcemanager.resourcemanageregoslot.callbackcontainerstatechg(resourcemanageregoslot.scala:525) org.apache.spark.deploy.master.resourcemanager.resourcecallbackmanager$$anonfun$callbackcontainerstatechg$1.apply(resourcemanager.scala:158) org.apache.spark.deploy.master.resourcemanager.resourcecallbackmanager$$anonfun$callbackcontainerstatechg$1.apply(resourcemanager.scala:157) scala.option.foreach(option.scala:236) org.apache.spark.deploy.master.resourcemanager.resourcecallbackmanager$.callbackcontainerstatechg(resourcemanager.scala:157)",   "serversparkversion" : "1.6.0",   "submissionid" : "driver-20160420043532-0027-6e579720-2c9d-428f-b2c7-6613f4845146",   "success" : true } driverstatus failed 

edit2:

finally problem had when submitting job has been solved creating brand new instance of spark service. job executes , finishes after few seconds.

but still receive error when trying download stdout , stderr files.

downloading stdout_1461156506108609180 % total    % received % xferd  average speed   time    time     time  current dload  upload   total   spent    left  speed   0    90    0    90    0     0     61      0 --:--:--  0:00:01 --:--:--   125 failed download workdir2/driver-20160420074922-0008-1400fc20-95c1-442d-9c37-32de3a7d1f0a/stdout stdout_1461156506108609180  downloading stderr_1461156506108609180 % total    % received % xferd  average speed   time    time     time  current dload  upload   total   spent    left  speed   0    90    0    90    0     0     56      0 --:--:--  0:00:01 --:--:--   109 failed download workdir2/driver-20160420074922-0008-1400fc20-95c1-442d-9c37-32de3a7d1f0a/stderr stderr_1461156506108609180 

any ideas?

i found old spark-submit trying retrieve stdout , stderr workdir folder ...

failed download workdir/driver-20160418191414-0020-5e7fb175-6856-4980-97bc-8e8aa0d1f137/stdout     stdout_1461024849908170118 

while new (downloaded yesterday) spark-submit trying download them workdir2 folder ...

failed download workdir2/driver-20160420074922-0008-1400fc20-95c1-442d-9c37-32de3a7d1f0a/stdout stdout_1461156506108609180 

the folder in use fixed variable ss_spark_work_dir initialized in spark-submit

if [ -z ${ss_spark_work_dir} ];  ss_spark_work_dir="workdir2"; fi # work directory on spark cluster 

i changed value workdir , works now. have downloaded new (today) spark-submit bluemix site , problem has been fixed. variable points workdir.

so, if fails, sure got last spark-submit script bluemix.