I am on a Windows, these are the relevant values:
(base) C:\Users\User>echo %JAVA_HOME%
C:\Users\User\Downloads\OpenJDK8U-jdk_x64_windows_hotspot_8u392b08\jdk8u392-b08
(base) C:\Users\User>echo %PYSPARK_SUBMIT_ARGS%
--jars C:\spark-3.5.0-bin-hadoop3\jars\graphframes-0.8.3-spark3.5-s_2.12.jar pyspark-shell
(base) C:\Users\User>echo %SPARK_HOME%
C:\spark-3.5.0-bin-hadoop3
(base) C:\Users\User>pyspark --version
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 3.5.0
/_/
Using Scala version 2.12.18, OpenJDK 64-Bit Server VM, 1.8.0_392
Branch HEAD
Compiled by user ubuntu on 2023-09-09T01:53:20Z
Revision ce5ddad990373636e94071e7cef2f31021add07b
Url https://github.com/apache/spark
Type --help for more information.
(base) C:\Users\User>java -version
openjdk version "1.8.0_392"
OpenJDK Runtime Environment (Temurin)(build 1.8.0_392-b08)
OpenJDK 64-Bit Server VM (Temurin)(build 25.392-b08, mixed mode)
(base) C:\Users\User>javac -version
javac 1.8.0_392
the error starts when trying to create a new SparkSession using the Session Builder :
PySparkRuntimeError(
pyspark.errors.exceptions.base.PySparkRuntimeError: [JAVA_GATEWAY_EXITED] Java gateway process exited before sending its port number.
Code : spark = SparkSession.builder.appName("YourAppName").getOrCreate()
Please help!
I have tried setting up Pyspark and graphframes on colab and it works fine but i cant seem to set it up on my local system. What am i doing wrong?