scmeta rewrite in Clojure

[niklas-heer]

I have tried to rewrite the scmeta tool from Crystal to Clojure, and personally, I find the Clojure code to be much more simple and beautiful, but I have one question and one challenge.
I did this because I wondered how a functional approach to the scmeta tool would look like, and since I'm handling mainly data Clojure seemed like a good fit. The second reason was just curiosity on my end if it would be hard to do or not, and it was actually pretty straightforward to do.

Both source codes as a comparison

Clojure code

speed-comparison/scmeta/scmeta.clj

Lines 3 to 105 in 310f79c

	(require '[docopt.core :as docopt]
	'[babashka.process :refer [sh]]
	'[cheshire.core :as json]
	'[clojure.string :as str])
	(defn wrap-as-time
	"Takes a given `time` in seconds and returns it as a time String.\n
	This is useful for working with dataframes. (appends an s)"
	[time]
	(str time "s"))
	(defn parse-int
	"Takes a given String [s] and returns an Integer."
	[s]
	(Integer/parseInt (re-find #"\A-?\d+" s)))
	(defn read-hf-file
	"Takes a hyperfine.json result file and returns a flattened hash-map."
	[file]
	(-> file
	(slurp)
	(json/parse-string true)
	:results
	first))
	(defn read-pi-file
	"Takes a pi.txt file and returns the content as a trimmed String."
	[file]
	(-> file
	(slurp)
	(str/trim-newline)))
	(defn pi-accuracy
	;; https://github.com/niklas-heer/speed-comparison/issues/78#issuecomment-1292708468
	"Takes a given number for Pi and returns the accuracy.
	The higher the number the better:
	- pi-accuracy(3.14) = 3.29
	- pi-accuracy(3.145) = 2.96
	- pi-accuracy(3.1415) = 4.53
	- pi-accuracy(3.1415926535897) = 13.5"
	[pi]
	(let [x (Double/parseDouble pi)]
	;; Formular: accuracy(x) = -log10(abs(1-x/pi))
	(- (Math/log10 (abs (- 1 (/ x Math/PI)))))))
	(defn get-version
	"Takes a command as `cmd` which gets executed as a shell process.
	The command should return some version number like: `1.0.0`
	The second parameter `index` determines which version number to return, should there be multiple.
	Returns a version number as String."
	[cmd index]
	(-> cmd
	(sh)
	:out
	;; re-seq needs the input as the last argument so we use "thread-last" (->>)
	(->> (re-seq #"\d+(\.\d+)+"))
	(nth index)
	first))
	(defn generate-json [arg-map]
	(let [lvi (parse-int (arg-map "--lvi"))
	lv (get-version (arg-map "--lv") lvi)
	pi (read-pi-file (arg-map "--pi"))
	hyperfine (read-hf-file (arg-map "--hf"))
	accuracy (pi-accuracy pi)]
	(json/generate-string {:Language (arg-map "--ln")
	:Version lv
	:Command (hyperfine :command)
	:CalculatedPi pi
	:Accuracy accuracy
	:Mean (wrap-as-time (hyperfine :mean))
	:Median (wrap-as-time (hyperfine :median))
	:Min (wrap-as-time (hyperfine :min))
	:Max (wrap-as-time (hyperfine :max))
	:Stddev (wrap-as-time (hyperfine :stddev))
	:UserTime (wrap-as-time (hyperfine :user))
	:SystemTime (wrap-as-time (hyperfine :system))
	:TimesPerRun (hyperfine :times)
	:ExitCodesPerRun (hyperfine :exit_codes)}
	{:pretty true})))
	(def usage "SCMeta.
	Usage:
	scmeta [options]
	Options:
	--ln=<name> Name of the language.
	--lv=<cmd> Command to get the language version.
	--lvi=<int> If there are multiple versions for `lv`, select the index [default: 0].
	--hf=<json> Hyperfine JSON file path.
	--pi=<txt> Path to pi.txt file.
	--out=<json> Path to write the output to")
	(docopt/docopt usage
	*command-line-args*
	(fn [arg-map]
	(->> arg-map
	(generate-json)
	(spit (arg-map "--out")))))

Crystal code

speed-comparison/scmeta/src/scmeta.cr

Lines 1 to 155 in ad10524

	require "json"
	require "option_parser"
	require "math"
	# Third party dependencies. See shard.yml
	require "json_on_steroids"
	NAME = "scmeta"
	VERSION = "1.1.1"
	upcase = false
	lang_name = nil
	lang_version = nil
	lang_version_match_index = 0
	lang_version_cmd = nil
	# Files
	hyperfine_file = nil
	pi_file = nil
	output_file = nil
	OptionParser.parse do |parser|
	parser.banner = "Usage: #{NAME} [arguments]"
	parser.on("--lang-name=LANGUAGE", "Specifies the name of the programming language") { |lang| lang_name = lang }
	parser.on("--lang-version=COMMAND", "Specifies the command to get the version of the programming language") { |cmd| lang_version_cmd = cmd }
	parser.on("--lang-version-match-index=NUMBER", "Specifies the match index to select from the command to get the version of the programming language. Default 0") { |num| lang_version_match_index = num }
	parser.on("--hyperfine=FILE", "Specifies the path to the JSON result of hyperfine") { |file| hyperfine_file = file }
	parser.on("--pi=FILE", "Specifies the path to txt file output of hyperfine") { |file| pi_file = file }
	parser.on("--output=FILE", "Specifies the path where to output all metadata as JSON") { |file| output_file = file }
	parser.on("-v", "--version", "Show verison") do
	puts "#{NAME} #{VERSION}"
	exit
	end
	parser.on("-h", "--help", "Show this help") do
	puts parser
	exit
	end
	parser.invalid_option do |flag|
	STDERR.puts "ERROR: #{flag} is not a valid option."
	STDERR.puts parser
	exit(1)
	end
	end
	def run_cmd(command : String)
	cmd_array = command.split(" ", 2)
	cmd = cmd_array[0]
	args = cmd_array[1].split
	stdout = IO::Memory.new
	stderr = IO::Memory.new
	status = Process.run(cmd, args: args, output: stdout, error: stderr)
	if status.success?
	{status.exit_code, stdout.to_s}
	else
	{status.exit_code, stderr.to_s}
	end
	end
	def set?(variable, msg)
	if !variable
	puts "#{msg}"
	exit(1)
	end
	end
	def get_version(text, match_index)
	matches = text.scan(/\d+(\.\d+)+/)
	begin
	matches[match_index].try &.[0]
	rescue ex
	puts ex.message
	puts "These are the caputed versions.\n"
	matches.each_with_index do |match, i|
	puts "#{i}: #{match.try &.[0]}"
	end
	exit(1)
	end
	end
	# Check if we got all variables set
	set?(lang_name, "Please provide the language name!")
	set?(hyperfine_file, "Please provide the hyperfine JSON file!")
	set?(pi_file, "Please provide the pi txt file!")
	set?(output_file, "Please provide the output JSON file!")
	set?(lang_version_cmd, "Please provide the language version command!")
	# Read JSON results from hyperfine
	hyperfine = File.open("#{hyperfine_file}") do |file|
	JSON.parse(file).on_steroids!
	end
	# https://github.com/niklas-heer/speed-comparison/issues/78#issuecomment-1292708468
	def pi_accuracy(input : String)
	accuracy = 1 - ( input.to_f / Math::PI)
	-Math.log(accuracy.abs, 10)
	end
	# Calculate pi accuracy
	computed_pi = File.open("#{pi_file}") do |file|
	file.gets_to_end.delete("\n")
	end
	accuracy = pi_accuracy(computed_pi)
	# Get the version by the provided command
	version_raw = run_cmd(lang_version_cmd.to_s)
	if version_raw[0] == 0
	lang_version = get_version(version_raw[1].to_s, lang_version_match_index.to_i)
	else
	puts "Getting language version failed!"
	puts version_raw[1]
	exit(1)
	end
	# Returns a given number as seconds
	def to_timedelta(number)
	"#{number.to_s}s"
	end
	# Build new JSON
	metadata = JSON::OnSteroids.new
	metadata["Language"] = lang_name
	metadata["Version"] = lang_version
	metadata["Command"] = hyperfine.dig("results.0.command")
	metadata["CalculatedPi"] = computed_pi
	metadata["Accuracy"] = accuracy
	metadata["Mean"] = to_timedelta(hyperfine.dig("results.0.mean"))
	metadata["Stddev"] = to_timedelta(hyperfine.dig("results.0.stddev"))
	metadata["UserTime"] = to_timedelta(hyperfine.dig("results.0.user"))
	metadata["SystemTime"] = to_timedelta(hyperfine.dig("results.0.system"))
	metadata["Median"] = to_timedelta(hyperfine.dig("results.0.median"))
	metadata["Min"] = to_timedelta(hyperfine.dig("results.0.min"))
	metadata["Max"] = to_timedelta(hyperfine.dig("results.0.max"))
	metadata["TimesPerRun"] = hyperfine.dig("results.0.times")
	metadata["ExitCodesPerRun"] = hyperfine.dig("results.0.exit_codes")
	# Write results to output_file as pretty JSON
	File.open("#{output_file}", "w") do |file|
	json = JSON.parse(metadata.to_json)
	file.print json.to_pretty_json
	end
	puts "Successfully created metadata\n"
	puts "Language: #{lang_name} (#{lang_version})"
	puts "Output: #{output_file}"

Crystal downsides

Preface: I'm new to Crystal and Ruby.
While there seems to be some support for functional programming, I've seen some discussions on the Crystal language like adding support for more functional language features similar to Elixir, but it seems that not many have made it in yet. It feels to me like Crystal is primarily an Object-Oriented programming language (OOP).
That is not a bad thing in itself, but I would like to do more functional code to get better at it.

The question

If I were to choose the Clojure implementation, would it make it less accessible for contributions?

The challenge

I have chosen Go (in the past) and Crystal for the speed-comparison tooling because they are both languages capable of producing portable static binaries. Meaning that I can build them in a first step with earthly and copy the binary into each language container with no additional installation requirements. Which is a pattern which works great.

Solution one: build a native Clojure binary using GraalVM (seems hard)

For Clojure, I either would have to build a native binary with GraalVM which from the brief look I took seems quite challenging to do.

Solution two: rework how the result JSON is built

The other option would be to use the filesystem, let every run write the results into a file, and then generate the combined JSON afterwards. The version command could also just output the stdout into a file, same goes for the language name.
But that would render having a separate language besides Python somewhat pointless because the analysis step in the end could just do that as well.

Final thoughts

I feel that what I gain in simplicity on the implementation side of scmeta would be eaten up by making the build or comparison execution more complex. 🤔
Maybe another solution could be to just use the analysis step with Python and write functional Python code.
I'm genuinely interested what others think of this.