2 years ago
#46125
Aman
Passing a google_credential.json file as an environment variable in AWS EMR Cluster
I am fairly new to AWS EMR. So i am trying to get data from google cloud bigquery into aws via emr. I have managed to do this in lambda but since size is huge need to switch to emr. Using Library
from google.cloud import bigquery
Now need to pass GOOGLE_APPLICATION_CREDENTIALS.json file as environment variable in emr. So how should i go about it. A step by step direction are much appreciated. Second question is can i do this with glue too. I already tried with connectors available online but the data i got was very messed up. So i wish to try with this library only but no idea how to proceed in glue with this. Thanks for any help.
python
amazon-web-services
gcloud
amazon-emr
aws-glue
0 Answers
Your Answer