Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compressed CSV is not working for loading data #49891

Open
roshanlabh opened this issue Aug 16, 2024 · 1 comment
Open

Compressed CSV is not working for loading data #49891

roshanlabh opened this issue Aug 16, 2024 · 1 comment
Labels
type/bug Something isn't working

Comments

@roshanlabh
Copy link

Steps to reproduce the behavior (Required)

Attempt to query a compressed CSV file in Google Cloud Storage using SELECT * FROM FILES function in StarRocks:

SELECT * FROM FILES (
  "path" = "gs://ctap_auto/1723627416-1723627416-Push-Impressions-20240814-0-6.csv.gz",
  "format" = "csv",
  "compression" = "gzip",
  "csv.column_separator"=",",
  "csv.row_delimiter"="\n",
  "csv.enclose"='"',
  "csv.skip_header"="1",
  "gcp.gcs.service_account_email" = "...",
  "gcp.gcs.service_account_private_key_id" = "...",
  "gcp.gcs.service_account_private_key" = "..."
)
LIMIT 100;

Expected behavior (Required)

The query should successfully read and return data from the compressed CSV file stored in Google Cloud Storage.

Real behavior (Required)

The query fails with the following error message:

Access storage error. Error message: not supported format: csv

StarRocks version (Required)

  • 3.2.8-759cc78
@roshanlabh roshanlabh added the type/bug Something isn't working label Aug 16, 2024
@chenminghua8
Copy link
Contributor

Limits
File external tables must be created in databases within the default_catalog. You can run SHOW CATALOGS to query catalogs created in the cluster.
Only Parquet, ORC, Avro, RCFile, and SequenceFile data files are supported.
You can only use file external tables to query data in the target data file. Data write operations such as INSERT, DELETE, and DROP are not supported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants