Failure to integrate h2leveldb with Mix.

[ghost]

I added this git repo as a dependency in my Mixfile, and tried to compile it. It gave me this result:

uniaika 18:58 ~/Elixir/databases/foo ❯❯ mix compile
==> qc (compile)
==> h2leveldb (compile)
/home/uniaika/code/Elixir/databases/foo/deps/h2leveldb/c_src/build_deps.sh: line 119: cd: /home/uniaika/code/Elixir/databases/foo/_build/dev/lib/snappy: No such file or directory
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
ERROR: Command [compile] failed!
** (Mix) Could not compile dependency :h2leveldb, "/home/uniaika/.local/bin/rebar compile skip_deps=true deps_dir="/home/uniaika/code/Elixir/databases/foo/_build/dev/lib"" command failed. You can recompile this dependency with "mix deps.compile h2leveldb", update it with "mix deps.update h2leveldb" or clean it with "mix deps.clean h2leveldb"

Do you have any idea about what's going on?

[tatsuya6502]

Thank you for the report. I have never tried h2leveldb from Elixir but the error message implies c_src/build_deps.sh get_deps did not get called before Mix called ./rebar compile. c_src/build_deps.sh get_deps is written in pre_hooks -> get-deps section of rebar.config, so it will get called when you invoke ./rebar get-deps. Perhaps Mix is not calling it?

h2leveldb/rebar.config

Line 6 in 95358d9

{'get-deps', "c_src/build_deps.sh get_deps"},

[ghost]

I'm opening an issue on the elixir-lang repository.

[ghost]

Alright so Eric gave us a tip to fix this

[tatsuya6502]

Alright so Eric gave us a tip to fix this

Thanks. I just saw it. I will give it a try.

[tatsuya6502]

Started to work on this on raw-deps branch.

Commit: 65722d5

• Added raw dependencies to rebar.config
• Removed get-deps and update-deps commands from the custom build script.

Tomorrow, I will try to build it from Elixir Mix.

[ghost]

Thanks for your work :)

[tatsuya6502]

Pushed another commit 4d5a89f to raw-deps branch.

When Mix runs rebar complie, $REBAR_DEPS_DIR is set to a wrong directory. I added extra code to the custom build script to workaround the problem.

[tatsuya6502]

@annwenn When you have time, can you please update your mix.exs with the following changes and see if it works?

• (For now) use raw-deps branch for h2leveldb.
• Add override dependencies for HyperLevelDB and snappy.

So the deps of your mix.exs will look like the followings.

  defp deps do
    [{:h2leveldb, git: "https://github.com/leveldb-erlang/h2leveldb.git", tag: "raw-deps"},
     {:HyperLevelDB, git: "https://github.com/leveldb-erlang/HyperLevelDB.git",
      compile: false, app: false, override: true},
     {:snappy, git: "https://github.com/leveldb-erlang/snappy.git",
      compile: false, app: false, override: true},

     # other deps ...
    ]
  end

You will need such override dependencies because (I think) Mix is not handling Rebar's raw dependencies correctly when running or testing the project. Without them, mix test and iex -S mix will fail with the following errors:

% mix test
Unchecked dependencies for environment test:
* HyperLevelDB (git://github.com/leveldb-erlang/HyperLevelDB.git)
  could not find an app file at _build/test/lib/HyperLevelDB/ebin/HyperLevelDB.app. 
  This may happen if the dependency was not yet compiled, or you specified the wrong 
  application name in your deps, or the dependency indeed has no app file (then you
  can pass app: false as option)
* snappy (git://github.com/leveldb-erlang/snappy.git)
  could not find an app file at _build/test/lib/snappy/ebin/snappy.app.
  This may happen if the dependency was not yet compiled, or you specified the wrong
  application name in your deps, or the dependency indeed has no app file (then you
  can pass app: false as option)
** (Mix) Can't continue due to errors on dependencies

I will report this problem to Elixir project. Hope they will fix it.

If you have any other problem, please let me know. (also tell me the Elixir and Erlang versions and the operating system you are using.)

I tried it with:

• Arch Linux and FreeBSD
• Elixir 1.2.5
• Erlang/OTP 18.3.3

p.s.
Make sure you use '/tmp/ldb.leveldb1' for the DB path in Elixir instead of "/tmp/ldb.leveldb1".

[ghost]

Appart from the error, everything seems to go fine.
I'm using

• Elixir 1.3.0-rc.0 thanks to Kiex.
• Erlang 18.3
• Funtoo Linux

The basic CRUD operations described in README.md work fine.

iex(1)> :h2leveldb.start_link []
{:ok, #PID<0.129.0>}
iex(2)⋅❯ db = '/tmp/leve.db'
'/tmp/leve.db'
iex(3)⋅❯ {:ok, db_ptr} = :h2leveldb.create_db(db)
{:ok, #Port<0.5862>}
iex(4)⋅❯ :ok = :h2leveldb.put(db_ptr, "foo", "bar")
:ok
iex(5)⋅❯ {:ok, "bar"} = :h2leveldb.get(db_ptr, "foo")
{:ok, "bar"}
iex(6)⋅❯ :h2leveldb.delete(db_ptr, "foo")
:ok
iex(7)⋅❯ :h2leveldb.delete(db_ptr, "foo")
:ok
iex(8)⋅❯ :h2leveldb.get(db_ptr, "foo")
:key_not_exist

[tatsuya6502]

Thank you for testing the raw-deps branch. I am glad to hear that it is　working for you.

Please give me few more day to get the raw-deps branch merged into the master branch. I will leave this issue open until then.

[ghost]

Alright :) Thanks for your work!!

[tatsuya6502]

Action Items

• Remove the warning message about $REBAR_DEPS_DIR from c_src/build_deps.sh
• Update README for Elixir Mix users
• Remove the hot-backup feature from the develop branch by doing the followings:
  • Create a topic branch from the develop branch
  • On develop branch, revert commit b7e84bb, which is for the hot-backup feature
• Merge raw-deps branch into the develop branch
• Merge develop branch into the master branch
• File a bug about Rebar raw deps problem to Elixir project