Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make profiles resolve recursively? #19

Open
iku000888 opened this issue Jan 25, 2019 · 6 comments
Open

Make profiles resolve recursively? #19

iku000888 opened this issue Jan 25, 2019 · 6 comments

Comments

@iku000888
Copy link

iku000888 commented Jan 25, 2019

Old behavior of :duct.core/include key

config foo.edn

{:foo/foo 1}

config bar.edn

{:bar/:bar 2
 :duct.core/include ["foo.edn"]}

config.edn that includes bar.edn

{:duct.core/include ["bar.edn"]}

And prep'ing config.edn would result in

{:foo/foo 1
 :bar/bar 2}

Trying something similar with #duct/include

#duct/include assumes the expanded config map will be under a profile key, so simple replacing :duct.core/include with #duct/include will not do.

e.g.

config foo.edn

{:foo/foo 1}

config bar.edn

{:bar/bar 2
 #duct/include "foo.edn"}

config.edn that includes bar.edn

{:some/profile #duct/include ["bar.edn"]}

does not recursively resolve.

Some more contexts from slack convo

selection_146

selection_147

@weavejester
Copy link
Contributor

Thanks. Could you elaborate on your use-case a little further? How exactly are you splitting up your configuration, and why?

@iku000888
Copy link
Author

The one I had is a datomic-connection component that is used by a kafka-streams component that is used by the app.

The datomic-connection configuration has to be provided by the app as during development I would use datomic-peer and in production I would use datomic cloud which we would switch via profiles.

Some more reasons doing this style were that we have several micro services of this manner and config files were getting really huge so it was a good way to split things up in a hierarchical manner.

The app is something like this;

App repository

Both files have the config for :datomic/connection

  • datomic-peer/datomic-connection.edn (in the classpath when datomic-peer profile)
  • datomic-cloud/datomic-connection.edn (in the classpath when datomic-cloud profile)

config.edn

{:duct.core/include ["stream-processor.edn"]}

Kafka streams lib repository

stream-processor.edn

{:duct.core/include ["datomic-connection.edn"]
 :kafka-stream-component/stream {:datomic-connection #ig/ref :datomic/connection
                                                           ;; and other configs....
                                                           }
 }

@weavejester
Copy link
Contributor

While this isn't a general solution, if you're just choosing the values of a single key, then you could write:

{:datomic/connection #duct/include "datomic-connection.edn"
 :kafka-stream-component/stream {:datomic-conneciton #ig/ref :datomic/connection}}

And then in datomic-connection.edn:

{:uri "datomic://..."}

Or whatever configuration you want.

@danielytics
Copy link

I found this issue while looking for a similar solution, so let me explain my use case:

I wanted to include migrations in a separate config file, both the duct.migrator.ragtime :migrators list and the migration components themselves, as the main config file is getting large and I feel that migrations should live in a separate file. I can of course include the :migrators list easily, but how do I also merge all of the individual migration components into the base profile? The old :duct.core/include would have done the trick, but I'm unsure how to solve it with just the reader literal.

I was thinking maybe I could create a migrations profile and include the migration components into that, but I don't understand how to use profiles other than the defaults. Can I make that profile always be merged in (regardless of dev or prod) and is there any documentation for using custom profiles? I didn't find any.

@weavejester
Copy link
Contributor

I was thinking maybe I could create a migrations profile and include the migration components into that, but I don't understand how to use profiles other than the defaults. Can I make that profile always be merged in (regardless of dev or prod) and is there any documentation for using custom profiles?

There needs to be more documentation on this, I agree! If you take a look at your current configuration, you'll see that there are three profiles:

  • :duct.profile/base - which is always included
  • :duct.profile/prod - included only in production
  • :duct.profile/dev - included only in development

So what do you do if you want to create your own profile? Just inherit from the one whose behavior you want in your config.edn file:

{[:foo.profile/migrations :duct.profile/base] #duct/include "foo/migrations"
 ...}

If you want your config file to be less cluttered, you can move the inheritance to src/duct_hierarchy.edn:

{:foo.profile/migrations [:duct.profile/base]}

And then in your config.edn you can just write:

{:foo.profile/migrations #duct/include "foo/migrations"
 ...}

@danielytics
Copy link

Thank you very much, that's perfect!! Thanks for a great library, by the way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants