Skip to content

Latest commit

 

History

History
78 lines (47 loc) · 5.23 KB

csv-import-export.mdx

File metadata and controls

78 lines (47 loc) · 5.23 KB
Raw
title description image ogImage author date published tags slug
Bring (and take) your own data with CSV import and export
Simplified data import and export in the Xata UI to bring, manage, and retrieve your own data
src alt
Bring and take your own data with CSV import and export
Richard Gill
08-28-2023
true
csv
import
export
csv-import-export

We know how important bringing your own data is when getting started with Xata. Often you’ve already got your own data and you don’t want to waste time manually creating a schema from scratch and then writing the code to import the data.

Love it or hate it, CSV is simply the world's most popular data format. While the option to import CSV files via the Xata CLI has always been part of our offering, we've taken things a step further.

Simple at the surface, the devil is really in the details if you want to have a great CSV import experience - which we do! Here are some things we considered:

  • There is no well defined standard and lots of variations. For example there are different separators and header can be present or missing.
  • We wanted to support large files (millions of rows and GBs of data), meaning rows need to be streamed and processed in batches, while ensuring there are no memory leaks in the browser.
  • Importing with the correct data types is important. For example, a column might use numbers in the first few records, but then have a string value further in the file.
  • Treating and reporting errors in a user-friendly way is important. No on wants to attempt importing millions of rows only to be hit with an unhelpful error.

Keeping these items in mind and taking into account valuable feedback from our user community, we've crafted a more user-friendly approach for data importing and exporting using the Xata UI. This means an easier, more intuitive way to work with your data, saving you time and making the whole process smoother.

As part of our launch week, we’re introducing a new CSV import UI experience, built from scratch, to make your Xata journey simpler - allowing you to automatically add new tables by importing your data directly into a Xata database. Also, we’ve added an export feature that allows you seamlessly download your data as a CSV file.

Preview before you import

To get started click Import CSV in the sidebar:

Import a CSV file from the sidebar

Browse or drag your CSV file (read the docs to see the formats we support).

Xata will preview the first 1000 rows in the file and intelligently guess the column types based on the data in your CSV.

In some cases the column types might not be what you want, in which case you can change them in the drop-down and the preview will refresh. You can also rename columns.

Selecting the column types

Errors are highlighted in red. When you’re happy with how the preview looks, you can...

Start the import

Importing large files can take a while, so be sure to leave your browser tab open while the import happens.

Any errors encountered while importing rows will be displayed in real-time. You'll have the option to download a summary of all these errors once the import is finished.

Showing the CSV import errors

If something isn’t right you can stop the import and go back to the preview screen to tweak your settings.

There is no cap on the size of the files you can upload - we’ve successfully imported 10 million rows in a 1GB CSV file.

Open source import

We’ve open-sourced @xata.io/importer our core code which parses CSV files - if you're interested in building on top of our importer, we invite you to contribute.

Exporting to CSV

We’ve also added the highly requested feature to download a Xata table as a CSV file.

To download your table click the Actions button at the top of your table:

You can download your data as a CSV file

There are two options for downloading a CSV file:

  • Export full table to CSV: This exports all the rows from your table. Links only include the ID. You might find this useful for creating a quick back up, or exporting data into another system (or right back into another Xata table).

  • Export current view to CSV: The will export a CSV which looks exactly the same as your current table view (including filters, columns and sorts). You might find this useful to share data with someone or importing a subset of the rows into a spreadsheet for analysis.

Happy importing and exporting!

We’d love your feedback on this feature, if you have any suggestions, questions, or issues reach out to us on Discord or follow us on Twitter. We'd love to hear from you and keep you updated on the latest at Xata.