Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: bulk CIDs import #2307

Open
wants to merge 48 commits into
base: main
Choose a base branch
from

Conversation

MattWong-ca
Copy link
Contributor

@MattWong-ca MattWong-ca commented Nov 30, 2024

Introduces bulk import of CIDs using a text file, as requested in #2173

Users can upload a text file with a list of CIDs (and an optional name right after), as shown below:

bafkreibm6jg3ux5qumhcn2b3flc3tyu6dmlb4xa7u5bf44yegnrjhc4yeq
QmawceGscqN4o8Y8Fv26UUmB454kn2bnkXV5tEQYc4jBd6 barrel.png
QmbvrHYWXAU1BuxMPNRtfeF4DS2oPmo5hat7ocqAkNPr74 pi equals.png
  • Click Bulk import from Import dropdown and a modal to select a text file will pop up
  • Frontend validation is done on each line of the text file using isIPFS.cid(), if any line fails then user won't be able to import it. Otherwise the name of the selected file will be shown
  • Added new function doFilesBulkCidImport in actions.js responsible for going through text file and adding each CID
  • Added stories.js file
  • Didn't include a CLI tutor mode command since it seemed more complex than the other ones already present, but happy to revisit if needed!

Testing

  • Manually tested to ensure it works in both the root folder and subfolders
  • Tried using a text file with improper formatting, which didn't let me import
  • NOTE: the most number of CIDs I tested with in a file was 91, at which there was a slight lag before all the files were imported

Demo

Bulk.Import.Final.Demo.mp4

@MattWong-ca
Copy link
Contributor Author

this is great, thanks so much! Only change I can see that is needed is the removal of prop-types, and having the new BulkImportModal be a functional typescript component.

Also, we should use kebab-case-filenames instead of CamelCase. This repo hasn't been fully updated, but that's where we're going

Thanks @SgtPooki, just finished addressing all the comments!

lidel and others added 25 commits December 13, 2024 20:12
…pfs#2305)

* fix(explore): chunked files

This includes latest ipld-explorer-components with fix from
ipfs/ipld-explorer-components#462

also bumped kubo and caniuse and non-breaking audit suggestions

* fix(files): Inspect via context menu

Closes ipfs#2306
https://github.com/ipfs/ipfs-webui/actions/workflows/ci.yml?page=4&query=is%3Asuccess 
are usually under 10-20 minutes
if something takes longer it will likely take ages and then fail, 
so better to fail faster, allowing user to retry release
no need to start node and open outgoing connections
github CI may be punishing us by throttling egress
## [4.4.1](ipfs/ipfs-webui@v4.4.0...v4.4.1) (2024-11-30)

 CID `bafybeiatztgdllxnp5p6zu7bdwhjmozsmd7jprff4bdjqjljxtylitvss4`

 ---

### Bug Fixes

* add lithuanian to languages.json ([ipfs#2293](ipfs#2293)) ([40c512b](ipfs@40c512b))
* analyze script doesn't persist stats.json ([ipfs#2290](ipfs#2290)) ([dbbdd70](ipfs@dbbdd70))
* **explore:** browsing chunked files and inspecting via context menu ([ipfs#2305](ipfs#2305)) ([0412970](ipfs@0412970)), closes [ipfs#2306](ipfs#2306)

### Trivial Changes

* **ci:** add CAR file directly to cluster ([ipfs#2304](ipfs#2304)) ([e2ae110](ipfs@e2ae110))
* **ci:** no replication factor when pinning - use cluster's default ([ipfs#2302](ipfs#2302)) ([81b8f29](ipfs@81b8f29))
* **ci:** set cluster pin timeout to 30m ([4b8fc00](ipfs@4b8fc00))
* **ci:** udpate artifact actions to v4 ([ipfs#2292](ipfs#2292)) ([305908f](ipfs@305908f))
* **ci:** use repo in offline mode ([eaf63ed](ipfs@eaf63ed))
* pull new translations ([ipfs#2291](ipfs#2291)) ([bfe7e40](ipfs@bfe7e40))
* pull transifex translations ([ipfs#2296](ipfs#2296)) ([502abd4](ipfs@502abd4))
* pull transifex translations ([ipfs#2303](ipfs#2303)) ([89c094b](ipfs@89c094b))
* size-related labels in Files screen ([ipfs#2295](ipfs#2295)) ([49019d4](ipfs@49019d4))
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: lidel <lidel@users.noreply.github.com>
@MattWong-ca
Copy link
Contributor Author

Hey @SgtPooki, wanted to bump this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants