How To Create Generic DataSet Function in Spark Scala

Creating a generic function that turns a Dataset of type row in scala to Dataset of some typed classes !

Below I have created a function named createTypedDataset which takes in dataset of type row objects and converting that into typed dataset.

import spark.implicits._

def createTypedDataset[T <: Product : TypeTag](ds: Dataset[Row]): Dataset[T] = {
  ds.as[T]

}


Continue reading my posts as I will be giving more such examples to teach you how to write more generic code.