Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Outflux fails silently when Influxdb server has query-timeout defined #91

Open
wiebeytec opened this issue Dec 17, 2021 · 0 comments
Open

Comments

@wiebeytec
Copy link

Running outflux, it consistently stops after about 900 seconds:

2021/12/08 16:06:35 pipe_479_ext: Extracted 83655000 rows from Influx
2021/12/08 16:06:35 pipe_479_ext: Extracted 83670000 rows from Influx
2021/12/08 16:06:35 pipe_479_ing: Complete. Inserted 83670000 rows.
2021/12/08 16:06:35 All pipelines finished
2021/12/08 16:06:35 Migration execution time: 940.285 seconds

But, it's far from complete. It managed to load about a day of data, but it should load 6 months.

Our Influx server has a query-timeout of 900 seconds, so this seems to be because of that.

I can't even work around it by batching using multiple calls to outflux, because only DropAndCreate works, because of querying the Timescale version 2 schema fails.

Expected/requested behavior:

  1. At least detection of the error. According to the documentation, Influx 1 should say: ERR: query timeout reached.
  2. Ability to batch the reading in x number of rows, or time range, etc.

Outflux version 0.2.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant