Simple spark environment setup in windows OS.
1. spark3.3
2. hadoop3
3. Java11
4. Python3.7
5. maven3.6
6. VSCode1.72
1. Download all the files in this repository and copy the root folder[Apache] to C:\.
2. Execute the environment variable setup bat [C:\Apache\Spark3.3\tools\environment_variable_setup.bat].
1. In CMD line, you can run your sparkSQL file like :
[python C:\Apache\Spark3.3\tools\spark-3.3.0-bin-hadoop3\bin\spark-submit.py C:\Apache\Spark3.3\source\example.py]
2. You can also develop and run the script file in VScode's debug mode [SparkSubmit]
The setted VScode : [C:\Apache\Spark3.3\tools\VSCode-win32-x64-1.72.0\Code.exe]
1. Fork it (https://github.com/vekee/D-Tools.git)
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create new Pull Request
DUAN DAHAI