How to access AWS S3 from Spark (Google Dataproc)
How to access AWS S3 from Spark (Google Dataproc). procedure Spark Configuration The following Spark and Haddop settings will allow you to read and write AWS S3 files from Spark. Load the following AWS-related jar files into Spark aws-java-sdk-bundle-xxxx.jar hadoop-aws-xxxx.jar Set the following p…
