Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem statements for a hackathon #15

Open
answerquest opened this issue Apr 27, 2018 · 15 comments
Open

Problem statements for a hackathon #15

answerquest opened this issue Apr 27, 2018 · 15 comments

Comments

@answerquest
Copy link
Collaborator

answerquest commented Apr 27, 2018

25.10.18: Edit:

Link to the Slides shown onscreen at the meetup


Gathering problem statements for a hackathon.
Guidelines:

  • One problem statement per post. If you have multiple, then make separate posts.
  • Clearly show skillsets required to qualify for participating in the task.
  • Provide reference links if any at bottom.
  • Make it small, not big. We want people to actually finish the task and put out the solution within a half-day or whatever the hackathon's action duration is. If the task is looking big, break it down into multiple tasks.
  • We want to solve real world problems with the task. Hence, provide a realistic use case scenario or better a real life need (mention who needs it)
@answerquest
Copy link
Collaborator Author

answerquest commented Apr 27, 2018

Best route mapping for a Delivery Van

  • A van sets out from an origin lat-long location and has a list of lat-longs to visit and then has to return home.
  • The program should churn out the quickest route, by road, to reach all the points and return back to the origin point.

Probable function call:

routeFinder( { Origin lat:xx.xx, lon:yy.yy } , [ 
     {id:'A1', lat:xx.xx, lon:yy.yy}, 
    { ...(Destination points)... } 
    ]
);
  • Arguments passed as json. Destination is array of jsons.
  • Returns a link to a google directions map, or returns a route shape, with the destination points marked with the id key passed.
  • Another possible output : A PDF or JPEG map ready for printing with the route drawn and destination ID's labeled.

Specifics

We could use google directions, or any open source routing mechanism : whichever is easier. The google map API requirements won't be too high; this program needs to be run only one or a few times in a day.

Reference:

Real world application : MeraFarmer, a social enterprise doing organic foods delivery service in Pune, is looking for this kind of solution to optimize their electric cart's delivery routes and maximize their area and customer coverage (while not running out of battery!), and also want to send their customers the route map with estimated time of arrival.

Sample Data, of actual locations of a delivery run:

latitude,longitude,name
18.538181,73.784732,Origin
18.530648,73.810429,A
18.577498,73.813334,B
18.562004,73.807477,C
18.560634,73.812726,D
18.562955,73.818957,E
18.551968,73.809506,F
18.554745,73.800882,G
18.556018,73.794711,H
18.558204,73.792850,I
18.562328,73.780350,J
18.571946,73.782667,K
18.577635,73.776756,L
18.570844,73.782323,M
18.583362,73.782861,N
18.593711,73.793340,O
18.596440,73.777531,P

@answerquest
Copy link
Collaborator Author

Live location and trip tracking through phone app

  • An app like GPSLogger on a phone does the tracking.
  • The app is configured to send the logged location to a web server / http addr etc every 5 seconds or 10 seconds or so.
  • The trip so far and current location gets published on a map on a web link.
  • Metrics like time taken to be published.
  • Coloring of route based on speed, speed color legend

Skillsets needed

  • Javascript
  • Leaflet or Google maps API for mapping. Preferably Leaflet.
  • Python or other server-side script
  • Web server endpoint making
  • Familiarity with a GPS logging app that will be good for the job.

Details discussion

  • The app could be sending a GET request (or an HTTP request.. AFAIK they're the same) to a URL with the lat, long as URL arguments.
    Ex: http://123456.herokuapp.com/API/postLocation?lat=xx.xxx&lon=yy.yyy
  • An HTTP endpoint (possibly a Tornado web class/function) receives the API call, logs the lat & long and timestamp, returns a success message as callback.
  • The webpage makes a different GET request, and gets all the trip details so far. It refreshes the data every 10 seconds or so.
  • Next day trip : when server receives a fresh ping from the app after a long period of time (say 4 hrs) then it starts recording a new trip and will give only the new trip to the webpage.
  • There is be a history function on the webpage to review past trips.
  • In python3, TinyDB would be a good database engine for this. It stores in a local .json file.
  • There can be a time slider to take the trip to different time to see where the position was at a particular time.

@answerquest
Copy link
Collaborator Author

Desktop shortcut creator for pyinstaller-created windows executable version of a python app

Use case scenario: python script : name.py

  • Standalone windows executable is created using pyinstaller at the path /dist/name/name.exe .
  • The exe has supporting DLLs etc that help it run, at the same place.
  • The supporting data files, HTMLs, folder structure etc that the original python program uses is all at .
  • We don't want to move these into ./dist/name/, and we don't want to move all those DLLs etc up. We want to keep executables and data separate, and run things from where they are, which will also be usefull later on for maintenance, re-building new versions etc.
  • We can't do --onefile to bundle all the DLLs etc into the .exe for a variety of reasons including incompatibility with win7 & lower and threat of false positive virus claim by antivirus programs.
  • Placing a shortcut to the .exe file at and keeping the "Star in" field blank in the shortcut's properties, seems to do the job. The exe is able to use its supporting DLLs for getting started, and then the program's data files are accessed at only because "Start in" is blank.
  • Using relative shortcut path in the "Target" field of the shortcut seems to work: %windir%\system32\cmd.exe /c start dist\GTFSmanager\GTFSmanager.exe
  • But this works only if the shortcut is placed at . If we place the shortcut in desktop, then there is no relative path that we can use from Desktop.. the shortcut needs the absolute "Target" and "Start in" paths. And this will change from system to system.
    What we want to do

Aim

  • Create a batch (.bat) file or something that goes places a shortcut to the application on the desktop.
  • The shortcut should have "Target" field like : D:\blah\blah\<top>\dist\name\name.exe (depending on where the program folder is)
  • ..and "Start in" field as: D:\blah\blah\<top>

References

@answerquest
Copy link
Collaborator Author

Local video files management with tagging

Skills needed

  • Python on Windows
  • Files and folders handling
  • Video file handling, conversion (ffmpeg)

Task points

  • Given a configured relative or absolute path, scan all files and folders under it, and pick files with video extensions.
  • In a web or GUI interface, recreate the files/folders structure, with thumbnails.
  • Enable the user to edit filenames
  • Tag videos, arrange in hierarchical categories
  • ... which is used for: searching and displaying things by these tags / categories. Including wordpress-style searching categories where searching the parent shows up its children too but not the other way around.
  • Duplicate folders, files finder (design or incorporate existing)
  • Use video processing to create low-D versions of videos for archiving and bigger versions can then be "sacrificed for the greater good"
  • On importing, recreate the nested folders structure as a category hierarchy and assign them accordingly to the videos.
  • Don't bother moving videos from their folders when categories etc change later on. Preserve the categorization in the local db.
  • Enable selecting multiple videos and exporting to a new folder.

References

Real World Application

  • Adventures Beyond Barriers, a non-profit that works on inclusion for disabled, needs a solution to manage their burgeoning videos collection.
  • Potential use by practically any team that needs to manage many videos.

@answerquest
Copy link
Collaborator Author

answerquest commented May 4, 2018

Map based data collection with variable key-value pairs

  • Have a location picker
  • A table or something where key-value pairs can be entered, similar to when ediing on OpenStreetMap.
  • Presets in a dropdown that will add some fixes key-value pairs and populate keys to be given values by the user.
  • Data is submitted to a noSQL DB.
  • A view section on the site shows the data thus collected on a map
  • Enable filtering of data by key-value pairs, similart to when using Overpass Turbo to extract data from OpenStreetMap.
  • See Project: NoSQL database #8

@answerquest
Copy link
Collaborator Author

Select layers in a multi-layered PDF map and generate high-res image with just the selected layers

See #12

@answerquest
Copy link
Collaborator Author

Scrape and visualize bank accounts data from PMJDY site

See #10

@answerquest
Copy link
Collaborator Author

answerquest commented May 4, 2018

GUI tool for unpivoting data (and pivoting)

@answerquest
Copy link
Collaborator Author

answerquest commented May 4, 2018

Ironing out messy Location data

Problem statement

Tasks

  • Show data in both map and tabular / similar form.
  • Use filters at both sides to filter data down to the group we need to iron out.
  • Enable manual selection for inclusion into the group.
  • Enable to select one lat-long value and impose it on all the entries.
  • Similarly for other metadata. But not all together. Allow user to iron out one column at a time.
  • Use geohashes or similar techniques for clustering. See Geohash idea for Bus stops (and other location redundancy) de-duplication #14.

Real world application

  • Apply to Pune PMPML Bus stops data released on Pune Open Data Portal.

@answerquest
Copy link
Collaborator Author

Visualizing hierarchical json of Pune's Annual Budget

See https://github.com/datameet-pune/datameet-pune.github.io/wiki/Budget-data-visualizer

@pankajp
Copy link

pankajp commented May 14, 2018

Local video files management with tagging

A starter:
https://besjournals.onlinelibrary.wiley.com/doi/abs/10.1111/2041-210X.12892
http://vixen.readthedocs.io/en/latest/overview.html

@answerquest
Copy link
Collaborator Author

answerquest commented May 14, 2018

Linguistic matching to merge datasets

Tamhini >> matches >> Taminhi
Velhe >> matches >> Vhele
तासगांव >> matches >> Tasgaon

Looking up and merging across datasets by doing linguistic matches that can tolerate spelling differences, and possibly even cross language barriers.

For more details see https://github.com/datameet-pune/datameet-pune.github.io/wiki/Match-the-Following

@answerquest
Copy link
Collaborator Author

Legacy font to Unicode office document converter

Let's have a tool that takes a word/excel/powerpoint file and converts all the ShreeDev, Shivaji etc fonts in it to Unicode. The converters are already made; they just need to be applied while preserving formatting.

See: https://github.com/datameet-pune/datameet-pune.github.io/wiki/Legacy-Devnagri-to-Unicode-font-converter-for-formatted-documents-like-word%2C-spreadsheet%2C-presentation

@answerquest
Copy link
Collaborator Author

answerquest commented May 19, 2018

@screwgoth
Copy link

If anyone wants to help take it forward, the source code for the Django server of the "Live location and trip tracking through phone app" problem is here:
https://github.com/screwgoth/live-location-tracker

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants