Skip to content

SETL-0.4.1

Compare
Choose a tag to compare
@qxzzxq qxzzxq released this 13 Feb 15:06
· 117 commits to master since this release

Changes:

  • Changed benchmark unit of time to seconds (#88)

Fixes:

  • The master URL of SparkSession can now be overwritten in local environment (#74)
  • FileConnector now lists path correctly for nested directories (#97)

New features:

  • Added Mermaid diagram generation to Pipeline (#51)
  • Added showDiagram() method to Pipeline that prints the Mermaid code and generates the live editor URL 🎩🐰✨ (#52)
  • Added Codecov report and Scala API doc
  • Added delete method in JDBCConnector (#82)
  • Added drop method in DBConnector (#83)
  • Added support for both of the following two Spark configuration styles in SETL builder (#86)
    setl.config {
      spark {
        spark.app.name = "my_app"
        spark.sql.shuffle.partitions = "1000"
      }
    }
    
    setl.config_2 {
      spark.app.name = "my_app"
      spark.sql.shuffle.partitions = "1000"
    }

Others:

  • Improved test coverage