Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write some UI tests #322

Open
5 of 52 tasks
jonko0493 opened this issue Jan 14, 2024 · 2 comments
Open
5 of 52 tasks

Write some UI tests #322

jonko0493 opened this issue Jan 14, 2024 · 2 comments
Assignees
Labels
tests Used for issues related to adding or modifying tests
Milestone

Comments

@jonko0493
Copy link
Member

jonko0493 commented Jan 14, 2024

We're gonna need UI tests, which will be especially useful when we do things like #321.

We will need to make a TestAssets.zip file and download that onto the machines to provide images/scripts/etc. to compare against the ones we create during the tests.

Some ideas:

  • Backgrounds
    • Import a background from a test asset, re-export and see if it's the same -- the files themselves will be different, but appium has a GetImagesSimilarity method that would likely work for this
    • The crop/resize tool should also be tested somehow
    • CG titles should be tested to see if they persist / end up in EXTRA.S
  • BGMs
    • Have test asset audio and a waveform image of that audio, replace the audio and generate a waveform image, then compare the two images for similarity
      • Just having pre-gen'd audio waveforms for all our sounds and comparing against them would be a great unit test, actually. Worth thinking about.
    • Check that extract produces a file of non-zero size, I suppose. Could do the same thing with replace.
    • One thing that would work is Extract -> Replace -> Restore -> Extract resulting in two extracted files of the same size/content, so we could test that workflow and hit a couple of scenarios
    • Not sure if there's a way we could test that the play/stop buttons work? I'm not sure how to check if sound is playing on a test machine (appium certainly doesn't have a built in method to do that) -- this StackOverflow answer implies this could be done through a library called CSCore. Worth looking into. Looked into it, this library is ancient. The way to do this on Windows is with NAudio's WASAPI implementation, which I've done, but it's very difficult to do it on macOS so I've ignored that problem for now.
    • Check that changing the BGM title persists (change it, close the tab, reopen the tab, see that the text is still the same). We could also check for the presence of EXTRA.S on disk after this change is made (and since we're designing from scratch, we could even compare it against a test asset or static string to see if the change propagated)
    • Probably ought to do something about loop properties and volume changing, but I'm not sure exactly how to go about it. Maybe waveform comparisons against test assets, but not sure.
  • Character Sprites
    • Check a couple of them to see that both frames and gifs export properly (compare against test assets)
  • Characters
    • Check that changing the character name changes the tab name
    • Check at least a few of the default colors to see if the color on the screen matches appropriately (could try to do this with screenshots but might only work on mac)
    • Change the colors and names of a couple of characters, then save. Compare the generated image (B87.png) against a test asset to make sure they are the same
    • Check that the voice font link works and links to the correct voice font; check that changing it links to a different (correct) voice font
    • Check that adjusting all these properties (including text timer) propagates between tab close/open
  • Chibis
    • Not sure if we can compare gifs, so I'm not sure how to check if animations are the same here. At the very least, would be worth checking to see that import/export of frames work and that exporting a gif results in a non-zero size file
    • Could check to see that changing the poses results in different images (again, probably only on mac tho)
  • Group Selections
    • For now, just check that some of the links work
  • Maps
    • Not a lot we can test here. If we make it so we can export the maps with (esp with the pathing map & starting point overlays) we could then do image comparison. Could also do these as unit tests, tho
  • Places
    • Check to see that the tab title changes on save
    • Check to see that the image produced is the same as a test asset on text change
  • Puzzles
    • Same as before, I'd just check that the links work. We could allow exporting the singularity image and then test that it matches a test asset, but I don't think that's particularly interesting to do
  • Scenario
    • Save a couple of SCENARIO.S files to the test assets and compare against them
    • Move some commands around
    • Add a command, select it, change some properties
    • Delete that command
    • Repeat the above two options for every scenario command
    • Delete all the commands with the rocket ship
  • Scripts
    • Truly so many things we can/need to do here. Same as scenario, compare against saved test assets
    • We should add the ability to export the preview so we can compare it against test assets
    • Start with a rocket ship, save and check for empty script sections
    • Apply a script template
    • Add a command section
    • Attempt to construct a script in a sensible way using every command available and modifying things about them
    • Drag/drop, copy/paste commands to move them around
    • Adjust starting chibis
    • Adjust map characters (possibly add map characters to a script without them?)
    • Adjust choices
  • SFXs
    • Currently can only extract and play these, so not much to do here
  • System Textures
    • Test export/replace/replace with palette
  • Topics
    • Check that title edit changes tab title
    • Check that link to associated script works and that changing it changes the link
    • Check that the base time gain * the time percentage matches the seconds displayed
    • Check that changes propagate
    • Change a few, check against TOPIC.S
  • Voices
    • Same as BGMs
    • Subtitles! Maybe we should add the ability to export the subtitle preview here so that we can test that better
  • Renaming items should be tested somehow (not sure how rn)
  • Building should be tested. We should be able to make a bunch of changes and then check against the hash of a known output ROM.
  • Figure out how to test searching... probably could be done where we craft some queries and then check the first result opens a page we expect? or has a label we expect or something? idk lol
  • File -> Close Project and then attempt to reopen it from the quick launch menu
  • Save editing!
@jonko0493 jonko0493 added the tests Used for issues related to adding or modifying tests label Jan 14, 2024
@jonko0493 jonko0493 added this to the v0.3 milestone Jan 14, 2024
@jonko0493
Copy link
Member Author

We should also go through old platform specific bugs for mac/windows and add tests that would catch those cases

@jonko0493 jonko0493 modified the milestones: v0.3, v0.4 Jul 30, 2024
@jonko0493
Copy link
Member Author

As a result of #371, this is going to be kicked back a milestone as we'll have to rethink how we do these tests, most likely.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tests Used for issues related to adding or modifying tests
Projects
Status: No status
Development

No branches or pull requests

2 participants