I'm trying to apply Apache Ranger integrated EMR. So I installed Ranger Server (Admin Server) and EMR Plug-in for EMR-FS and EMR-SPARK.
EMR-FS works fine. However, it doesn't seem to work as intended in cases like EMR-SPARK. When I have checked the “Use for Spark table metadata” option when I create my EMR cluster, all databases that I create on spark-sql and Already created from Glue should be visible here and all databases that I create here should be visible on Spark.
Currently, only "Default" is searched when querying Database with Spark-sql.
spark-sql> show databases;
default
Time taken: 2.364 seconds, Fetched 1 row(s)
When configuring EMR with Ranger plugin, "Use the AWS Glue Data Catalog to provide an external Hive metastore for Spark" is this option also impossible?
If this is not possible, how can I search the Glue Catalog in Spark?