[Planning] - spectralworkbench.js / rails integration

[waridrox]

Working upon publiclab/spectral-workbench.js#219 to achieve the following:

• 1: Porting new version of spectralworkbench.js library into spectral workbench rails app. Updating dependencies, checking for breaking changes #671
• 2: Adding a /v2 route and view for the new capture integration. New capture code route and view at /v2  #660
• 3: Capture code re-factor Capture code refactor for new interface #672
• 4: Making sure the old capture code at /capture doesn't break. Capture code refactor for new interface #672
• 5: Making the new interface responsive. Capture code refactor for new interface #672
• 6: Creation of login prompt for user authentication to the app.
• 7: Expanding upon the capture integration to incorporate: Adding action for new interface for fetching recent calibration #694
  • saving of spectral data to the rails app (along with applied calibration). New save form partial for /capture/v2 #695
  • fetching recent calibrations and displaying them in a dropdown list to choose from. Adding action for new interface for fetching recent calibration #694
• 8: Camera switching in a dropdown. Camera switching in a dropdown list spectral-workbench.js#247
• 9: Warn users with unsaved changes when leaving/reloading webpages.
• 10: Improving accessibility across the app. Improving accessibility on the new interface at /v2 #701
• 11: Addition of tests/documentation wherever applicable. System tests for /capture /capture/v2 and /upload #702, Fixed dependency versions in docs #718

Open for suggestions!
CC: @jywarren @RuthNjeri @pydevsg

[jywarren]

This looks great. Your first two tasks listed look like a super place to begin!!

…

On Sat, Jun 5, 2021, 5:04 AM Mohammad Warid ***@***.***> wrote: Working upon publiclab/spectral-workbench.js#219 <publiclab/spectral-workbench.js#219> to achieve the following: - Porting new version of spectralworkbench.js library into spectral workbench rails app. - Replacing live capture code with the new code, thereby utilising the spectralworkbench.js library at a /capture/v2 OR /capture/new URL - Creation of login prompt for user authentication to the app. - Expanding upon the capture integration to incorporate: - saving of spectral data to the rails app (along with applied calibration). - fetching recent calibrations and displaying them in a dropdown list to choose from. - Horizontal scaling of x-axis on the live capture graph (in spectralworkbench.js) after choosing a calibration spectrum. - Warn users with unsaved changes when leaving/reloading webpages - Improving accessibility across the app - Addition of tests/documentation wherever applicable More suggestions are welcomed !! CC: @jywarren <https://github.com/jywarren> @RuthNjeri <https://github.com/RuthNjeri> @pydevsg <https://github.com/pydevsg> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#645>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAF6JYXOE2QQODV5GUBJO3TRHSDPANCNFSM46EIYP6Q> .

[jywarren]

You can point it at a commit, yes, or we can run npm publish? Let's do a commit first and I can show you the process for npm publish soon! Thanks!

…

On Mon, Jun 7, 2021, 6:41 PM Mohammad Warid ***@***.***> wrote: - @jywarren <https://github.com/jywarren>, it seems that the library on NPM at *https://npm.runkit.com/spectral-workbench <https://npm.runkit.com/spectral-workbench>* has not been updated to incorporate many of the recent changes that were added like the new distributable capture code at *https://github.com/publiclab/spectral-workbench.js/blob/main/dist/capture.dist.js <https://github.com/publiclab/spectral-workbench.js/blob/main/dist/capture.dist.js>* . - The library on NPM is reflective of the codebase at *https://github.com/publiclab/spectral-workbench.js/tree/630b0c24fb0e2849dad66a52eed125133c585f27 <https://github.com/publiclab/spectral-workbench.js/tree/630b0c24fb0e2849dad66a52eed125133c585f27>* . So, - should I just then pick the commits for the capture code at *https://github.com/publiclab/spectral-workbench.js/blob/main/dist/capture.dist.js <https://github.com/publiclab/spectral-workbench.js/blob/main/dist/capture.dist.js>* and proceed or - should I try to integrate the new capture code i.e. *https://github.com/publiclab/spectral-workbench.js/blob/main/dist/capture.dist.js <https://github.com/publiclab/spectral-workbench.js/blob/main/dist/capture.dist.js>* into *https://github.com/publiclab/spectral-workbench.js/blob/main/dist/spectral-workbench.js <https://github.com/publiclab/spectral-workbench.js/blob/main/dist/spectral-workbench.js>* itself so that all the library code lives under one file, since I think here yarn upgrade won’t simply work due to the fact that the new capture code is not present on the NPM release yet… — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#645 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAF6J65JFVTVWA3PYF3RUDTRVDJNANCNFSM46EIYP6Q> .

[waridrox]

So I managed to pull the recent changes by pointing the library to the main branch by the cmd bower install 'git://github.com/publiclab/spectral-workbench.js#main' --save and it worked! Also it seems there is no bower.json file present in the codebase. Will try to add it soon...

[jywarren]

ah, are we still using bower? I thought Spectral Workbench uses Yarn now, and the command would be yarn install. Could you try that as I think we are trying to deprecate bower usage!

Also - it may be best to fix it to a specific commit, as main will change quite often, and so merging to main could cause a test failure on the main branch of the parent app. Pinning it to a specific commit introduces an extra step within this repository to pull in the latest version (Dependabot will help automate this too!) and so we get a look in an independent PR at whether the latest version breaks the tests. Make sense? Thanks @waridrox !!

[jywarren]

Npm publish docs are here: https://docs.npmjs.com/cli/v6/commands/npm-publish/

Basically the first time or two i'd like to lead the publication process, then you can start doing it.

How familiar are you with semantic versioning? You can read about it here: https://docs.npmjs.com/about-semantic-versioning - remember the discussions in publiclab/image-sequencer#1751 about how this lets us communicate and coordinate around releases?

Thanks! How are things going overall?

[jywarren]

OK, so i've opened a PR bumping SWB.js to v0.2.0, as the last time it was published was 5 years ago! 😱

https://www.npmjs.com/package/spectral-workbench shows this. Once we merge it, we'll be able to publish a new version. A lot has been added but I don't believe it includes any breaking changes, so I consider it an 0.2.0 version number by semantic versioning. How does that sound?

[jywarren]

Awesome, i merged it and ran npm publish from the main branch. We are at v0.2.0 now!!!

[jywarren]

@waridrox we are still a little stuck on getting tests passing on #636 in github actions. Have you explored copying some of the setup from plots2 to get this running? This would unblock a TON of our dependency upgrades, and give your work this summer a lot more support!

[waridrox]

ah, are we still using bower? I thought Spectral Workbench uses Yarn now, and the command would be yarn install. Could you try that as I think we are trying to deprecate bower usage!

Hi @jywarren, thanks a lot for publishing the npm package 😄, things should be pretty simple now...unfortunately yarn install / npm install lead to an error. So I had to create a bower.json file to get the dependencies installed. But if we are moving away from bower then we should take a look at the licensing part of the dependencies as suggested in the error since as of now only the spectral-workbench.js library dependencies live inside the /node_modules/ folder.

How are things going overall?

Great!! Currently I'm trying to integrate the new capture interface at a capture/v2 URL...and this is the status so far 😅:

we are still a little stuck on getting tests passing on #636 in github actions

Sure I can take a look at this soon after working on the above...Thanks ✨

[jywarren]

Aha - ok, yes, so let's just ensure we use the right licensing string, we can reference:

https://github.com/publiclab/spectral-workbench.js/blob/4f29a95119af0aef7a4a2e131ec4815df475e1a8/package.json#L20

See how it's different here:

spectral-workbench/package.json

Line 6 in b764f35

"license": "GPL3",

Let's just change it to match!

[waridrox]

Hey everyone, I wanted to know if it is a good idea to place the required JS files from the library into the assets folder so that it gets utilised by the asset pipeline / should we point the JS files to the node_modules folder / should we look towards something like webpacker gem. What would be the best possible choice ?

Also regarding the package.json, could we likely create a new one since many of the dependencies have changed their names on NPM...
Here's a list of dependencies ported from the package.json file to direct to their latest compatible version for the app. I've checked the app with these dependencies and it doesn't appear to be breaking in functionality:

"dependencies": {
    "ace-builds": "^1.4.12",
    "bootstrap": "^3.4.1",
    "d3": "^3.5.4",
    "flot": "^0.8.0-alpha",
    "font-awesome": "^4.7.0",
    "getusermedia-js": "^1.0.0",
    "jasmine": "^3.7.0",
    "jasmine-ajax": "^4.0.0",
    "jquery": "^3.3.1",
    "jquery.steps": "^1.0.3",
    "junction": "^0.2.0",
    "leaflet": "^1.7.1",
    "moment": "^2.29.1",
    "nvd3": "^1.7.1",
    "spectral-workbench": "^0.2.0"
  }

Please voice your opinions regarding the same. Thanks
CC: @jywarren @pydevsg @RuthNjeri

[pydevsg]

Hi, @waridrox thanks for pointing them out.
Here are my views on your concerned questions.

1. It is always preferred to put the js into the assets folder. Also, put it inside javascripts folder.
2. Coming to package.json , you don't need to create a new package.json file. In order to put the latest version what you can do is "package-name":"*" this will directly install the recent versions.
   In case you want to add a specific version just write it and the latter will be installed.
   Let me know if you need any more help while integrating 😅

[jywarren]

Hi @waridrox - great questions.

It is always preferred to put the js into the assets folder. Also, put it inside javascripts folder.

This is true, but also Rails has evolved, messily, in terms of the integration of npm, compilers (sprockets earlier, webpacker later) and the Asset Pipeline. You can see that we customized the asset pipeline here just as in plots2, and we'll have to do it again for Rails 6 when that arrives. My feeling is that we should stick with what @alaxalves did in the big #499 #530 upgrades, and just make good plans for a bigger change in Rails 6, which is in the future.

Did you find that the package.json packages are no longer available? Are there some bower-era packages that still aren't represented in package.json, or are you saying some package.json packages have changed names -- and, which?

Thanks!

[jywarren]

Also, i'd like to ask if you could open a PR with the latest version so we can see how it does in Github Actions and GitPod... i think it'll help @pydevsg myself and you all stay synced! Thank you!!!

[jywarren]

Hi @waridrox i was going to say perhaps responsiveness could wait until the functional changes, but it looks like you've already started! 🎉

6: Creation of login prompt for user authentication to the app.

This one could be as simple as a link to /login. However, looking at the existing v1 /capture page, see how it offers to let you use it without a login, but encourages you to log in if you want to save? We should try a modal popup with the same kind of info.

[jywarren]

I'm going to close this up and open a new one for any remaining!