Spark

Documentation

• Spark 3.0.0 ScalaDoc

sbt configuration

libraryDependencies += "org.apache.spark" %% "spark-core" % "3.0.0"
libraryDependencies += "org.apache.spark" %% "spark-sql" % "3.0.0"

Environments

• Local Cluster

Examples

• Group by Key
• Pi

Demos

These files are literate Scala, and can be run using Codedown:

$ curl https://earldouglas.com/posts/scala/spark.md |
  codedown sbt > build.sbt

$ curl https://earldouglas.com/posts/scala/spark/local.md |
  codedown scala > Spark.scala

Group by Key

$ curl https://earldouglas.com/posts/scala/spark/group-by-key.md |
  codedown scala > Main.scala
$ sbt run
Name(List(John, Jane),Doe)
Name(List(Alice),Bob)

Pi

$ curl https://earldouglas.com/posts/scala/spark/pi.md |
  codedown scala > Main.scala
$ sbt run
Pi: 3.1415926635898956