Developer Log: Alexi

This will be used to keep track of my progress throughout the completion of this project

January: 12 hrs

start 1/2

Following the tutorial here

• Issue: can't connect using different topic name
• Solution: change device policy (it was only allowing pub/sub with "sdk/test/python"

success! I can turn an LED on and off using the MQTT test client

time: 2 hrs

Proposal redo: 2 hrs

playing around with mqtt: 1 hr

• Having issues SSHing into Pi in VSCode
  • updated and restarted

logged 1/14: 5 hrs

• Working on issue #10
  • time: 2 hrs
• Other stuff
  • gathering example vids: 1 hr
  • meeting with potential teammate: 1 hr
  • discussing with teammate; reviewing stock ticker: 1 hr
• Revamping drawing library: repo
  • nested git repos? stock ticker should build on drawing library. probably too much stuff to make this a gist. I think I'm just gonna start it over.
  • add python project and set up linting + tests with Actions: 1 hr
  • you can draw a board!: 1 hr

logged 1/27: 7 hrs

February: 14 hrs

Neopolitan

• lots of pylint and import errors: 1 hr
• redoing data transformation for writing symbols: 1 hr

Neo

• make code easier to understand: 1 hr
• install and configure pylint: 1 hr

Neopolitan

• add CL arguments for message display: 1 hr
• working on GH Actions error that is causing tests to fail: 1 hr
  • local pylint + pytest solved by removing __init__.py from src/
  • only run CI pytests in tests/ directory
  • add __init__.py with import sys; sys.path.append('src/') to tests/
• CI tests pass: 1 hr
  • couldn't merge branch because I was updating branch protection rules for the wrong repo...whoops

logged 2/3: 7 hrs

Neopolitan:

• did a bit more cleaning up: enabled scrolling speeds and padding if it scrolls itself out of data: 1 hr

Neo:

• reorganize mqtt connection code a bit and work on linter: 1 hr
  • probably doesn't work because aws_iot is not included in repo
• research git submodules to include aws_iot in the project without manually adding all the code to my repo
  • how-to
• figured out linting error: was downloading pylint twice (once directly, once in requirements): 1 hr
• get all changes merged in and resolve merge conflicts and clean up branches: 1 hr
• get linter to work: 1 hr
  • solution
  • also added a .bashrc alias to run my linting command on lint
• added pytest locally: 1 hr
  • need to configure secrets to get workflow to pass
• set up local secrets: 1 hr
  • linting is good. tests fail, but are set up (almost) correctly
  • env vars need to match case. duh.

logged 2/10: 7 hrs

Neo

• added an AWS MQTT test: 1 hr

Dibiasky

• start a (simple) react app and start looking into AWS authentication: 1 hr
• AWS Amplify tutorial: 1 hr
• Different AWS Amplify tutorial: 1 hr

March: 34 hrs

logged 3/5: 4 hrs

Dibiasky

• Got stuck on that tutorial and started looking for other ones: 1 hr
• Working on AWS Amplify tutorial - trying to attach policy: 2 hrs
  • Success!
  • Solution: needed to add wss://* and */mqtt to aws_pubsub_endpoint

• Migrating example code to repo: 1 hr
• Diagnosing ConnectionDisrupted errors: 1 hr
  • Solution: aws configure != sudo aws configure (run the latter)
  • IAM > Users > Allow user to IoTAttachPolicy
  • sudo aws iot attach-policy --policy-name 'policy' --target 'region:cognito' --output json
• Set up repo secrets and push to GH: 1 hr
• Read about designing MQTT topics: 1 hr
• Design IoT topics and abstract MQTT connection code: 1 hr

Neo

• Design IoT topics (same as above, just in Python): 1 hr

logged 3/10: 9 hrs

Neo

• Design IoT topics on Pi; debug errors to push: 1 hr

Dibiasky

• Working on designing IoT topics to get a response after publishing a command: 2 hrs
• Setting up CI; adding some tests; setting up branch protection: 1 hr
• Writing tests and dealing with Cognito/auth issues in Jest: 1 hr
• Set up linting workflow: 1 hr
• Fixing merge conflicts, messing around with React: 1 hr
• Messing around with React, debugging useState issues: 1 hr
• trying to figure out why it won't connect to Amplify: 3 hrs
  • Pi can still pubsub
  • still no idea
  • updated packages
  • checked that current PubSub version has no errors
    • potential issue with v5
  • solution: delete all aws-amplify files in node_modules and reinstall

logged 3/24: 11 hrs

Dibiasky

• working on Cognito test: 1 hr
  • Can still get connection status and cognito id without withAuthenticator
  • web app running or not doesn't affect it
  • still struggling, tabling for now

Neo

• Make it so webpage can run board operations: 1 hr

  • Allow all users to run neopixeltest with sudo without password
  • Add alias to neopixeltest in .bashrc
  • it kind of works, but not great (yet)
  • still need to input sudo password on first go

• Working with sudo permissions to run neopixeltest: 1 hr

  • solution: allow command to flash LEDs (using sudo) to individual user, not all users
  • I can trigger an LED animation from a webpage, that's pretty darn cool!

• Design a data topic: 1 hr

• req/res to get available Pi operations: 1 hr

Dibiasky

• webpage design, more repo organization: 1 hr
• make webpage wireframe match design: 1 hr
• working with flexbox: 1 hr

Neo

• broke power supply somehow. Now just powering off the Pi: 1 hr

Dibiasky

• refreshing on Redux: 1 hr
  • left off here

logged 3/31: 10 hrs

Official Start

Apr: 57 hrs

Dibiasky

• same as above, more Redux: 1 hr
• displaying log and current subscriptions: 2 hr
• display Pi connection status: 1 hr
  • aws iot list-things --output json
    • "An error occurred (AccessDeniedException) when calling the ListThings operation: User: arn:aws:iam::{} is not authorized to perform: iot:ListThings on resource: * because no identity-based policy allows the iot:ListThings action"
  • Potential fixes:
    • AWS IoT Events
    • Device Heartbeat
  • This is the solution

Neopolitan

• look into multiprocessing and events to allow listening for messages from Neo: 1 hr
• same area, research threading and events in Python: 1 hr
  • here's a look at what I have so far
• working with event queue to change displayed message: 1 hr
  • (not the best quality video)
• code organization: 1 hr
• added pylint, updated readme: 1 hr

Dibiasky

• researching Amplify with multiple devs article and amplify auth: 1 hr

Neo

• setting up laptop as a Thing: 1 hr
• adding env var for client id, needed to get tests to pass: 1 hr
  • running neopixeltest twice also breaks on laptop, even after bash says 'command not found' on first run
• trying to figure out why cl command stops execution on second run: 1 hr
  • works just fine using echo after taking off bashrc setup stuff
  • but neoptest then echo also breaks
  • as does echo neoptest
  • happens without sub.communicate as well
  • happens when command not found
  • shell=False same effect
  • issue was the '-i' arg

Dibiasky

• add connection state icon (instead of just text): 1 hr

Overall

• new teammate onboarding document: 2 hrs

logged 4/7: 16 hrs

Overall

• cleaned up project board and added a bunch of milestones: 1 hr
• added ideal roadmap to new developer onboarding document: 1 hr
• teammate onboarding discussion: 1 hr

Neo

• update readme: 1 hr
• document topic structure: 1 hr

Dibiasky

• added device connection status icon: 1 hr

Overall

• team meeting: 1 hr
• reading into Jest testing, helping teammate: 1 hr
• team meeting: 1 hr

logged 4/14: 9 hrs

Neopolitan

• document event-passing solution: 1 hr
• having issues SSHing into Pi: 1 hr
  • update and restart didn't solve
  • updating and restarting computer didn't work
  • downgraded ssh extension to 0.96 fixed it
• make the code work on the LED board: 1 hr
• swap between hardware and graphical board: 3 hrs
  • still having ssh issues in code. removed .vscode-server/ from Pi

hello_on_board.mp4

Overall

• team meeting, figuring out where to suggest we go next: 1 hr

Neopolitan:

• working with events code and writing tests, make event handling happen in main: 1 hr
  • git remote update origin --prune to get rid of deleted remote branches locally

Neo:

• change env vars to not be prepended with "Hubble": 1 hr
  • needed to add CLIENT_ID secret to workflow file

Neopolitan:

• work on making abstract classes to swap between graphical and hardware displays: 1 hr

Overall:

• helping teammate, reading about git: 1 hr

Neopolitan:

• redesigning available operations workflow: 2 hr
  • git config --global push.default current to autosetup remote
• making available operations buttons that can actually trigger the operations: 2 hr
  • + some project board organization

logged 4/21: 15 hrs

100 Hours

Overall:

• team meeting, discuss where we should go next, canvas updates: 1 hr

Dibiasky:

• working on getting a dynamic status of connected devices: 2 hr
  • lots of errors
  • was updating array state in useState wrong. solution
  • now issue of multiple additions to connected devices state

Neo:

• adding neopolitan as a submodule: 3 hrs
  • breaking a lot of stuff
  • import = module
  • Structuring Your Project - good article

Neopolitan:

• learning about Python modules and packages: 1 hr

Dibiasky:

• dynamic status of connected devices, lots of state update issues: 2 hrs
  • current state within a callback
    • useRef() seems to have fixed it
  • still having errors with saving a file not reloading page fully, probably due to my own misunderstanding

Neopolitan:

• still trying to figure out import error: 3 hrs
  • this relative imports in Python article might explain it
  • seems like you should always import from the top
  • seems like another good explanation
    • example repo
    • good example repo

Dibiasky

• working on making an available operations dropdown: 1 hr
  • using this component
• populate the dropdown with available operations: 1 hr

Neopolitan

• Python packaging tutorial: 1 hr
  • Python virtual environments
  • test package upload - might be able to see it before it deletes
• trying to find good examples to model my packaging system: 1 hr
  • not what I'm looking for
  • question: what happens if I make the code into a package as-is?
    • didn't work. was able to upload and install it though
• building and uploading to PyPi: 1 hr
  • works in virtualenv, needed to install pygame though

logged 4/28: 17 hrs

May: 88 hrs

Neopolitan:

• upload official package: 2 hrs
  • Check it out!

neopolitan_package_install.mp4

Neo:

• integrate neopolitan package: 2 hrs
  • AWS IoT MQTT Test Client not loading
    • still doesn't work after clearing history and restarting computer. opening in private window works. no idea why
  • can run neopolitan from dibiasky (still WIP though)

run_neopolitan_from_dibiasky.mp4

Overall:

• work on integrating the three repos together: 1 hr
• added logging to Neopolitan and Neo: 1 hr
• build routes between Dibiasky and Neo: 1 hr
• team meeting: 1 hr
• Continue integrating - work on sending data to update display: 4 hrs
  • almost ready to go
  • it works!

integration_mvp.mp4

• operations documentation: 2 hrs

  • Neopolitan Operations
  • Neo Operations
  • Dibiasky Operations

• documentation: 3 hrs

  • come up with more direction and pass off to teammate

neopolitan

• get package to work on Pi: 4 hrs
  • Python version issue
  • VSCode hangs when installing project dependencies (neopixel -> scipy,numpy)
  • try installing with default python version (3.7)
    • can't because it needs >=3.8
    • what happens if I install it with 3.7 and run it with 3.8?
      • it installs, so does scipy. but will it run??
      • it will not run.
  • try installing troublesome packages on their own

neo

• update logging and workflow: 1 hr
  • shouldn't have messed with CI. If it ain't broke...

Dibiasky

• deployed to custom domain: 1 hr
  • www.alyoshenka.com

logged 5/5: 23 hrs

Dibiasky

• working on deployment, setting custom domain: 1 hr Neo
• Working with sub package logs: 1 hr

Neo

• Revisit project objectives, research AWS EventBridge for #3: 1 hr
• practice triggering lambdas from MQTT and MQTT from lambdas: 1 hr
• research setting up EventBridge Schedule from a Lambda: 1 hr

Overall

• team meeting: 2 hrs
• look into using Step Functions instead: 3 hrs

logged 5/12: 10 hrs

Neopolitan

• trying to get package to work: 4 hrs
  • dealing with Python versions
  • sudo apt-get install python3-pyaudio
  • sudo apt install libatlas-base-dev
  • neopixel dependencies
    • sudo pip3 install rpi_ws281x adafruit-circuitpython-neopixel
    • sudo python3 -m pip install --force-reinstall adafruit-blinka
    • pip3 install adafruit-circuitpython-neopixel
    • sudo python3 -m pip uninstall neopixel* issue was other neopixel library installed. it does something completely different
  • works, but not in venv
    • sudo apt-get install python3-dev python3-rpi.gpio didn't fix
    • pip install RPi.GPIO worked

Neo:

• trigger Step Function that invokes a Lambda from an IoT Topic: 1 hr

Dibiasky:

• add test button to schedule with an execution time: 1 hr

Neo:

• work on publish -> lambda -> step function -> lambda pipeline: 1 hr
  • code
• redesign topics and payloads: 2 hrs

Dibiasky:

• update to work with new payload design: 1 hr

Neo:

• brainstorm how to interface between React and DynamoDB: 1 hr

Dibiasky:

• add schedule button: 1 hr
• hook in schedule button to publish to MQTT: 1 hr

Neo:

• lambda to validate operation and add to database: 1 hr
• lambda to get scheduled operations database: 1 hr
• work on publishing to actual operation lambda: 1 hr
• document Scheduling Functionality: 1 hr

Overall:

• make sure everything is working to prepare for presentation videos: 1 hr
• make sure AWS code matches my documentation: 2 hrs
• test and debug entire scheduling pipeline: 2 hrs
  • It works! I can schedule board events and have them run in the future
• midterm presentation slideshow outline: 1 hr

logged 5/19: 23 hrs

Dibiasky:

• add timepicker and make datetime inputs update commandTime value: 1 hr

Neopolitan:

• update readme to document code usage: 1 hr
• add demo code and videos: 2 hrs
• update readme usage documentation: 1 hr

Dibiasky:

• allow deletion of scheduled items: 2 hrs

Neo:

• delete past items from database when its contents are queried: 1 hr

Dibiasky:

• try to get Amplify to work with DynamoDB: 2 hrs
• use AWS SDK instead: 2 hrs
• update datetime picker for better UX: 1 hr

Overall:

• working on midterm presentation: 2 hrs

Dibiasky:

• work on setting both update options and scheduled time: 1 hr

Overall:

• set up demo account for presentation: 1 hr
  • looks like it will take too much time to get it working for presentations tonight
• small fixes for presentation: 1 hr
• making diagrams: 1 hr
• team meeting: 1 hr
• add demo videos: 1 hr

Dibiasky:

• try to make it so that you can update a scheduled operation: 1 hr

Overall:

• team meeting, presentation prep: 1 hr
• midterm presentations: 2 hrs

Dibiasky:

• working on updating scheduled operation, little tweaks: 1 hr
• start integrating Victor's changes: 1 hr

Overall:

• re-prioritize important issues and make a game plan for tech showcase: 1 hr

Dibiasky:

• restructure folders: 1 hr

Overall:

• team meeting: 3 hrs

logged 5/26: 32 hrs

200 Hours

All Milestones Achieved

Overall:

• playing around with getting stock ticker to work: 3 hrs

Dibiasky:

• debugging issue where options disappear: 1 hr

Overall:

• some general brainstorming and playing around: 1 hr
• adding some more demos: 2 hrs
• made simple stock ticker display: 1 hr

Dibiasky:

• work on deleting step function along with scheduled operation: 3 hrs

Overall:

• team meeting: 1 hr
• messing around, fixing tiny things: 1 hr

Dibiasky:

• change connectionstatus to Redux: 1 hr
• trying to add demo user: 1 hr
• broke Auth - swapped auth and unauth roles because I didn't read carefully. fixed: 1 hr
  1. Make new user
  2. Verify new user
  3. Get user credentials
  4. Attach policy to identity ID
• connected devices when opening web app second; heartbeats kinda: 1 hr

Overall:

• project organization, presentation prep: 1 hr
• work on Topics and Payloads documentation: 1 hr
• messing around with stock ticker: 1 hr
• team meeting about tech showcase: 1 hr

Dibiasky:

• fix connection status issue: 1 hr

Overall:

• buy materials to make LED board case: 1 hr
• work on LED board case: 2 hrs

June: ? hrs

logged 6/2: 25 hrs

Overall:

• work on case: 1 hr
• work on wiring: 1 hr
• working on case: 2 hrs
• architecture diagrams: 1 hr
• tech showcase presentation writing: 1 hr

Neo:

• clean exit of stockticker: 1 hr

Overall:

• team meeting: 1 hr

Neo:

• small bug fixes: 1 hr

Overall:

• rebuilding display case: 3 hrs
• trying to fix wiring mistake: 1 hr
• documentation, working on final presentation: 1 hr
• rewiring, trying to fix board: 1 hr
• soldering and assembling board: 1 hr
• working on tech showcase presentation: 2 hrs
  • metrics

Neo:

• board clears on ctrl + c: 1 hr

Overall:

• test presentation: 1 hr

Dibiasky:

• team meeting and small fixes: 1 hr
• fixing routing issue in deployment: 1 hr

Overall:

• practice presentation: 1 hr

Dibiasky:

• allowing multiple pages in deployment: 1 hr
• working on About page content: 1 hr

logged 6/9: 25 hrs

Dibiasky:

• filling out the About page: 1 hr

Overall:

• small tweaks: 1 hr
• setting up board: 1 hr
• team meeting, small tweaks: 1 hr