Alright, I'm answering this for posterity. The issue was actually maven. It seems that I was using incompatible version of the two framework. Of course maven being maven, cannot detect this....
Environment variables and config files didn't work, but some manual code did: sc.hadoopConfiguration.set("fs.s3n.awsAccessKeyId", "PUTTHEKEYHERE")