Would you consider sharing the backend API method code needed to render the full report?

[SamuelDudley]

Hi,

Firstly, thank you very much for this project and putting all the frontend code out there 👍. I appreciate your work and your ongoing research in this field. Feel free to ignore the wall of text and skip to the TLDR; below.

I have forked and modified the frontend code to change up the reporting slightly with the intent of serving creepjs locally for repeated local browser plugin testing.
I have the frontend project (this repo) building and being served over https no problems (as a side note SubtleCrypto.digest() returns undefined when served over http which breaks creep.js). As creep.js runs, it makes a few API calls to https://creepjs-api.web.app which appears to be a GCP firebase project or similar. Notably the return from both https://creepjs-api.web.app/fp and https://creepjs-api.web.app/decrypt are required to fully render the results in the front end.

On first inspection I thought that all the calculations were occurring in the front end code and the backend API was just for tracking / persistence of visit data and that I could easily mock out the API by writing a couple of routes in a python flask app that I can also run locally. Now, having taken a look at the request arguments and the responses of these API calls its clear that the API methods are not trivial at all....

My notes on these API calls follow (bit of a brain dump, sorry!):
https://creepjs-api.web.app/fp returns quite a few values in its json response that I simply have no idea how the values were calculated. e.g.: bot, botHash (changed from the supplied value), botLevel, crowdBlendingScore (I think this is a fake value that is later updated after the later decrypt API call?), score, scoreData (looking at the git history this might have been moved to the API in this commit but the logic has changed?), shadow and shadowBits. The rest (there are not many more!) seem clear enough.

https://creepjs-api.web.app/decrypt seems to be performing a lookup of the values passed in the arguments against the cloudSamples data (but server side) and then making some assessments from those results. I think I understand generally what is happening but its unclear to me how some of the data passed to the API is used / impacts the score and how the scores for values not in the cloudSamples data are calculated (some scores seem to have lower weights if they cant be found and others affect the score a lot more). Lastly I'm unsure what pendingReview is representing in this response.

TLDR;

I'm attempting to run your project locally but I dont know the secret sauce happening in the https://creepjs-api.web.app/fp and https://creepjs-api.web.app/decrypt API calls (this might be by design?). I'm not at all interested in any of the save / load / track / other app logic, but purely the logic that is used to calculate the values returned to the front end to render the report.

Would you consider sharing the code / logic of these API calls either publicly or privately? I'm happy to do the work to port the code / get it working locally e.g. snippets would suffice.

Thanks,
Sam

dudley.samuel@gmail.com

[abrahamjuliot]

That is cool, the API caught your interest. I've decided to keep the server code private for a number of reasons, but I can give you the gist of what's going on. Some of this is touched on in the readme.

https://creepjs-api.web.app/decrypt

• both /decrypt and /fp run on GCP Firebase
• /decrypt captures metrics in a data model and renders the data to the cloud samples firestore
• The data model has a set of instructions on how to respond if the metric shows up again (reject or merge, timestamp, modify systems/devices, log reporter data, decide to require supervisor review or self-learn from patterns, and more)
  • newly discovered metrics start out with a very low score: this score can only improve if the metric echos a set of visits on distinguishable dates and via different reporters
  • newly discovered metrics that do not improve their score within 2 weeks get auto-deleted: if that metric returns later and it previously had made progress on improving the score, all that effort is erased
  • newly discovered metrics that are fake should find it difficult to maintain any level of trust over a long period of time
  • old metrics that aged 40 days from their last timestamp also get auto deleted: this prevents outdated browser version metrics from establishing a high crowd blending score (outdated activity should fail to blend in)
  • pendingReview started out as mostly a signal to me, but it's not necessary to show. Of the total metrics in the Firestore, this number of the total requires review. If this number starts to increase rapidly, that's a red flag. I have a number of other review flags that are not shown.
  • much of the logic in constructing cloud samples is kept secret to preserve the purity of the data and limit the need to review
• auto deletions & scheduled mass sample reads of the Firestore data and the sample API run on GCP Apps Script

https://creepjs-api.web.app/fp

• score is calculated from scoreData
  • the calculation logic is near the same as the history commit
• crowdBlendingScore (expounded on above) gets calculated via the decrypt API and then posted to the fingerprint data
• botHash is derived from select resistance patterns and computed on the front end
• bot is the percent of detections in the botHash
• botLevel is a label assigned to specific signals in the botHash
• if certain bot levels get detected, then the prediction goes into "locked" mode and all learning or updating data on the server gets shut down.

I hope this gives a better picture

[vis2021t]

I was looking for the same answer but one more thing caught up my eye which is the script we recive from :-

https://script.googleusercontent.com/macros/echo?user_content_key=znvovjjUBEB1WHjjlG5hbcQqBX8vNDpgEBfb0R9ovEyjEVfqZwx0afC0vHp2ihKCgCTx3nTF1VAZiMNEwiOkRmiXHCbZMUKJm5_BxDlH2jW0nuo2oDemN9CCS2h10ox_1xSncGQajx_ryfhECjZEnCiKSlQ8hv69EOVtcLYuHt1SlCc4BdULTIl5y8p6IXwI0GX_5RzxljD8-QABMnjlsiCbKCORCamKB07rGJaGc1Y50zFGQn3zBtz9Jw9Md8uu&lib=M5STdfrg6Nu03eVaFAkDuIEBcyd557MIy

Mind to give me a better View? regarding this dataset

[abrahamjuliot]

Right, so this is a collection of *non-stale and trusted data samples (auto computed and updated every Sunday). You can use JSON.parse on it. Actually, it's rendered in sessionStorage on load, so you can analyze in the console and de-structure the data like so:

data = JSON.parse(sessionStorage.samples)
blinkAudio = data.audio.Blink
fingerprints = Object.keys(blinkAudio).map(x => blinkAudio[x].id)

console.log(fingerprints.join('\n'))

*aging or low score samples auto removed

[vis2021t]

What do u mean exactly by supervisior review?

[abrahamjuliot]

Review, either by me or a function. It would involve determining if the data is poisoned and requires some cleanup.