Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide a no-brainer docker-compose build #80

Open
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

parisni
Copy link
Contributor

@parisni parisni commented Feb 25, 2024

After few years of hibernation, we tried an install from scratch and noticed few issues, leading to very painful experience. This PR solve the issues, and also provide a no brain docker-compose script to build the mimic-omop database. The build duration is less than 1 hour on a laptop, and it is compatible with both mimic demo or latest mimic 3 export.

Some of the major issues:

  • postgresql > 11 handle CTE differently leading to hang the ETL for eternity
  • indexing and analyze lack on concepts tables made the ETL also last way to long

Painful aspects:

  • manual instruction is quite vague on some parts
  • the R script needs local installation with no instructions
  • some scripts were missing
  • the makefile has broken references

@parisni parisni marked this pull request as draft February 26, 2024 09:08
@tompollard tompollard marked this pull request as ready for review March 12, 2024 18:36
@briangow
Copy link

@parisni , thanks for this!

I've attempted to run the Docker MIMIC-III demo build. I added the files to the mimic/data-mimicdemo and extras/athena folders as described in the docker/README. I get numerous errors and ultimately docker-compose up fails with exited with code 2. Please see the attached docker.log for details.

When initially setting things up, I also noticed that a lot of the links in the primary README are broken.

@parisni
Copy link
Contributor Author

parisni commented Apr 1, 2024

Hi @briangow thanks for your attempt. From the logs most of the errors you have are no big deal, for example missing schemas that are then created.

Eventually the data is created, as this example shows:

etl-1  | INSERT 0 31010
etl-1  | Time: 10634.488 ms (00:10.634)

But then this one retains my attention:

sh: 1: cannot create etl/Result/attribute_definition.csv.gz: Directory nonexistent

You might have to create a Result folder in etl to fix this. Tell me if this help, I would then fix this PR accordingly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants