inkweaver
/
mobileinsight-apptainer
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 2 Wiki Activity
mobileinsight-apptainer/with-spark.def

17 lines
344 B
Modula-2
Raw Normal View History

Revert "Removed spark implementation, use a docker-compose cluster for testing" This reverts commit 8943eceb93f7063b0af0d72924779699522f36d6.
2024-05-03 02:04:38 +00:00
Bootstrap: localimage
From: mobileinsight.sif
%post
# Update and install dependencies
export DEBIAN_FRONTEND=noninteractive
apt-get update
apt-get upgrade -y
apt-get install -y \
Add pandas to Spark dependency
2024-05-18 05:27:52 +00:00
python3-pandas \
Revert "Removed spark implementation, use a docker-compose cluster for testing" This reverts commit 8943eceb93f7063b0af0d72924779699522f36d6.
2024-05-03 02:04:38 +00:00
openjdk-8-jre-headless \
scala
Simplified spark version so pyspark can work
2024-05-03 02:27:04 +00:00
pip3 install pyspark==3.5.1
Revert "Removed spark implementation, use a docker-compose cluster for testing" This reverts commit 8943eceb93f7063b0af0d72924779699522f36d6.
2024-05-03 02:04:38 +00:00
# Cleanup
apt-get autoclean -y