2 years ago

#46125

test-img

Aman

Passing a google_credential.json file as an environment variable in AWS EMR Cluster

I am fairly new to AWS EMR. So i am trying to get data from google cloud bigquery into aws via emr. I have managed to do this in lambda but since size is huge need to switch to emr. Using Library

from google.cloud import bigquery

Now need to pass GOOGLE_APPLICATION_CREDENTIALS.json file as environment variable in emr. So how should i go about it. A step by step direction are much appreciated. Second question is can i do this with glue too. I already tried with connectors available online but the data i got was very messed up. So i wish to try with this library only but no idea how to proceed in glue with this. Thanks for any help.

python

amazon-web-services

gcloud

amazon-emr

aws-glue

0 Answers

Your Answer

Accepted video resources