Skip to content

devlog

Jump to bottom
Jack Wrenn edited this page Aug 25, 2018 · 5 revisions

Examplar Devlog

2018.08.25

Joe confirmed that the README was a little out of date and that a Redis database is necessary to get Google logins working.

Assignment template URLS work.. Implements template URLs in the form of #template=GDRIVEID, where GDRIVEID is the gdrive id of a public folder containing a file named template.arr. The contents of template.arr are loaded into the editor.

Unfortunately, this is overly simplistic because, unlike usual CPO, we need to support a consistent notion of an associated assignment. For a typical programming homework, we distribute a #share URL, which graduates to a student-specific #program URL once they make changes, and the provenance of the file is forgotten. For examplar, we still want the student to be able to save their changes, but we also need to retain a notion of associated assignment so we can run their file against wheats and chaffs.

So I'm now tackling this design issue and a few related ones (how do we store student content, how do we store analytics). My first instinct was the Google Drive Realtime API, which provides realtime-access to opaque gdrive-stored blobs. Unfortunately, it was recently deprecated. Ugh.

2018.08.24

First day of work! Shriram, Yanyan and I scoped out the project. I began development.

Project Scope

Tool Design

By the beginning of the semester, we will have a tool where:

  1. the student goes to a url, e.g., http://examplar.pyret.org/#template=<GDRIVEID>
  2. a CPO-like loads with a test sweep template for that assignment. The REPL is disabled.
  3. upon clicking "Run", the test sweep is run against a corpus of wheats and chaffs, and one of the following outputs:
    • if there are compilation errors, those are reported
    • otherwise:
      • if the user does not pass all wheats, some message indicating there was an issue with their test
      • otherwise:
        • if the user catches all chaffs, some message indicating they should move on to the rest of the assignment, and perhaps a note that some number of chaffs were not part of the exercise so they should expand their test suite for final submission
        • otherwise, the fraction of chaffs they caught

Usage Context

This tool will serve as a drop-in for the usual test sweep writing step of homeworks. Completion of a sweep using this tool will be a requirement, but submissions will not be directly graded. We will have some mechanism to keep track of when students seek help from TAs.

Study Design

Usage Monitoring

We'll monitor student progress developing their test sweeps via instrumentation.

Longitudinal Study

Hypothesis: Student test sweeps produced with this tool will be (1) finer-grain and (2) more effective than sweeps produced in previous semesters without in-flow feedback.

Initial Development Progress

Design

Students will navigate to a URL in the form of http://examplar.pyret.org/#template=<GDRIVEID>. GDRIVEID will point to a gdrive folder with this layout:

assignment_name
├ chaffs/
├ wheats/
└ template.arr

...where:

  • template.arr is a test template file for the assignment
  • chaffs/ and wheats/ are folders containing the compiled wheats and chaffs implementation modules

Implementation Progress

I familiarized myself with the gdrive API for listing folder contents. I'm confident that the above scheme will work just fine. Incidentally, listing the contents of even a public folder requires being logged in to a google account; this naturally enforces the requirement that students must login to use the tool!

I pulled CPO's horizon branch. The build process described in the README for the horizon branch result in a Pyret compilation error. I shifted gears and used master as a starting point, instead. This fixed the compilation error and I was able to run a local instance of CPO. However, after following the instructions for configuring google API credentials I've hit a wall: all logins fail on the login callback page which displays the error {"type":"DB error","error":{}}. The terminal I'm running the CPO server from displays a trace, too:

Error on getting user:  TypeError: Cannot read property 'hgetall' of null
    at process._tickCallback (internal/process/next_tick.js:61:11)
From previous event:
    at Object.getUserByGoogleId (/home/john/projects/examplar/src/storage/redis-store.js:5:14)
    at /home/john/projects/examplar/src/server.js:196:31
    at /home/john/projects/examplar/src/google-auth.js:102:9
    at /home/john/projects/examplar/node_modules/google-auth-library/lib/auth/oauth2client.js:154:5
    at Request._callback (/home/john/projects/examplar/node_modules/google-auth-library/lib/transporters.js:106:7)
    at Request.self.callback (/home/john/projects/examplar/node_modules/google-auth-library/node_modules/request/request.js:187:22)
    at Request.emit (events.js:182:13)
Authentication failure TypeError: Cannot read property 'hgetall' of null
    at process._tickCallback (internal/process/next_tick.js:61:11)
From previous event:
    at Object.getUserByGoogleId (/home/john/projects/examplar/src/storage/redis-store.js:5:14)
    at /home/john/projects/examplar/src/server.js:196:31
    at /home/john/projects/examplar/src/google-auth.js:102:9
    at /home/john/projects/examplar/node_modules/google-auth-library/lib/auth/oauth2client.js:154:5
    at Request._callback (/home/john/projects/examplar/node_modules/google-auth-library/lib/transporters.js:106:7)
    at Request.self.callback (/home/john/projects/examplar/node_modules/google-auth-library/node_modules/request/request.js:187:22)
    at Request.emit (events.js:182:13) TypeError: Cannot read property 'hgetall' of null
    at process._tickCallback (internal/process/next_tick.js:61:11)
From previous event:
    at Object.getUserByGoogleId (/home/john/projects/examplar/src/storage/redis-store.js:5:14)
    at /home/john/projects/examplar/src/server.js:196:31
    at /home/john/projects/examplar/src/google-auth.js:102:9
    at /home/john/projects/examplar/node_modules/google-auth-library/lib/auth/oauth2client.js:154:5
    at Request._callback (/home/john/projects/examplar/node_modules/google-auth-library/lib/transporters.js:106:7)
    at Request.self.callback (/home/john/projects/examplar/node_modules/google-auth-library/node_modules/request/request.js:187:22)
    at Request.emit (events.js:182:13)

I'm not sure what to make of these errors. The README doesn't mention anything about databases except for a brief note below the heading "Setting up your own remote version of code.pyret.org with Heroku" which isn't what I'm trying to do at this moment. The above trace mentions redis, however.

I've sought help on Slack, but, at the time of writing, haven't gotten any responses.

Clone this wiki locally