Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cloud Export integration, saving & display #1192

Merged
merged 57 commits into from
Mar 26, 2020
Merged
Show file tree
Hide file tree
Changes from 54 commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
e8b9cfc
initial exports per map display
jywarren Jan 10, 2020
dd3f8db
integration with new LDI v0.10.0, uploading status.json URL
jywarren Jan 10, 2020
fd1cda4
yarn.lock
jywarren Jan 10, 2020
60663b8
fix
jywarren Jan 10, 2020
c4d3a73
tab tweak
jywarren Jan 11, 2020
ddc1d04
console
jywarren Jan 11, 2020
4c6ccbd
LDI update to v0.11.0
jywarren Jan 11, 2020
94e7ad6
almost working!!
jywarren Jan 13, 2020
28dbecd
working!
jywarren Jan 14, 2020
ceb9d9c
docs link
jywarren Jan 14, 2020
30c2916
Update package.json
jywarren Jan 14, 2020
bd6f347
bump LDI to v0.11.5
jywarren Jan 14, 2020
a327ddc
fix popper
jywarren Jan 14, 2020
eb9b660
moved popper/bootstrap
jywarren Jan 14, 2020
f6b6d6f
new LDI
jywarren Jan 17, 2020
5b89522
fixed map_id in new map template
jywarren Jan 17, 2020
1aa5398
fixed popper to popper.js
jywarren Jan 17, 2020
5cc449b
include correct file
jywarren Jan 17, 2020
9048ae1
Tweak for precompiling assets
icarito Jan 21, 2020
44e6092
Move styling to stylesheets for login page (#1220)
gr455 Jan 26, 2020
1a00fe9
initial exports per map display
jywarren Jan 10, 2020
89c316b
integration with new LDI v0.10.0, uploading status.json URL
jywarren Jan 10, 2020
9b95652
yarn.lock
jywarren Jan 10, 2020
936abd3
fix
jywarren Jan 10, 2020
07bd82d
tab tweak
jywarren Jan 11, 2020
5afcada
console
jywarren Jan 11, 2020
3a0fbc7
LDI update to v0.11.0
jywarren Jan 11, 2020
29bf577
almost working!!
jywarren Jan 13, 2020
e02c20d
working!
jywarren Jan 14, 2020
99714be
docs link
jywarren Jan 14, 2020
46c55d7
Update package.json
jywarren Jan 14, 2020
85e41e2
bump LDI to v0.11.5
jywarren Jan 14, 2020
de4d269
fix popper
jywarren Jan 14, 2020
be48f15
moved popper/bootstrap
jywarren Jan 14, 2020
35e736c
new LDI
jywarren Jan 17, 2020
5fac7ca
fixed map_id in new map template
jywarren Jan 17, 2020
4bdddfc
fixed popper to popper.js
jywarren Jan 17, 2020
8865b44
include correct file
jywarren Jan 17, 2020
0de724d
Tweak for precompiling assets
icarito Jan 21, 2020
dac2125
Merge branch 'export-display' of github.com:jywarren/mapknitter into …
jywarren Jan 29, 2020
3b4a8a1
commenting out closure delay (function(){})()
jywarren Jan 29, 2020
68db9b6
popper properly included via bootstrap bundle
jywarren Jan 29, 2020
3055eb3
add SSL https://export.mapknitter.org
jywarren Mar 17, 2020
eff752a
"reorder for avoiding function hoisting" merge from unstable
jywarren Jan 29, 2020
7f18c45
Remove dropping database on redeploying container
icarito Mar 17, 2020
987beba
Fix missed doublequote
icarito Mar 17, 2020
e1a620d
Add asset compilation to Makefile
icarito Mar 24, 2020
1a710d8
Try exec instead of run
icarito Mar 24, 2020
f83471c
Move asset building
icarito Mar 24, 2020
af6abe7
Wrap yarn call in Makefile
icarito Mar 24, 2020
15a1e40
Mimic plots2 deployment config
icarito Mar 24, 2020
005534a
Find assets!
icarito Mar 24, 2020
c4ff692
Restore Makefile
icarito Mar 24, 2020
96300a4
Update Map.js
jywarren Mar 24, 2020
3926966
Update _cloud_exports.html.erb
jywarren Mar 25, 2020
42102a3
Update _cloud_exports.html.erb
jywarren Mar 25, 2020
4c4117e
Update Map.js
jywarren Mar 26, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,4 @@ redeploy-container:
docker-compose up --force-recreate -d
$(call wait_for_container)
docker-compose run -e "DISABLE_DATABASE_ENVIRONMENT_CHECK=1" --rm web bash -lc \
"bundle exec rails db:drop && \
bundle exec rails db:create && \
bundle exec rails db:schema:load && \
bundle exec rails db:migrate"
"bundle exec rails db:migrate"
4 changes: 2 additions & 2 deletions app/assets/javascripts/application.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,14 @@
//= require jquery-ujs/src/rails.js
//= require jquery-ui/jquery-ui.min.js

//= require bootstrap/dist/js/bootstrap.bundle.min.js

//= require blueimp-tmpl/js/tmpl.js
//= require blueimp-file-upload/js/vendor/jquery.ui.widget
//= require blueimp-file-upload/js/jquery.fileupload
//= require blueimp-file-upload/js/jquery.fileupload-process
//= require blueimp-file-upload/js/jquery.fileupload-ui

//= require bootstrap/dist/js/bootstrap.js

//= require leaflet-fullHash.js
//= require leaflet-providers/leaflet-providers.js
//= require leaflet-environmental-layers/dist/LeafletEnvironmentalLayers.js
Expand Down
86 changes: 79 additions & 7 deletions app/assets/javascripts/mapknitter/Map.js
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@

//(function() {
MapKnitter.Map = MapKnitter.Class.extend({

initialize: function (options) {
this._zoom = options.zoom || 0;
this._latlng = L.latLng(options.latlng);
this.map_id = options.map_id || 0;
this.readOnly = options.readOnly;
this.logged_in = options.logged_in;
this.anonymous = options.anonymous;
Expand Down Expand Up @@ -45,6 +46,7 @@ MapKnitter.Map = MapKnitter.Class.extend({
var downloadEl = $('.img-download-' + warpable.id),
imgEl = $('#full-img-' + warpable.id);

// this 'download' section can likely be dropped as Leaflet.DistortableImage now provides for such download itself
downloadEl.click(function () {
downloadEl.html('<i class="fa fa-circle-o-notch fa-spin"></i>');

Expand Down Expand Up @@ -418,6 +420,7 @@ MapKnitter.Map = MapKnitter.Class.extend({
var downloadEl = $('.img-download-' + warpable.id),
imgEl = $('#full-img-' + warpable.id);

// this 'download' section can likely be dropped as Leaflet.DistortableImage now provides for such download itself
downloadEl.click(function () {
downloadEl.html('<i class="fa fa-circle-o-notch fa-spin"></i>');

Expand Down Expand Up @@ -468,10 +471,7 @@ MapKnitter.Map = MapKnitter.Class.extend({
mode: 'lock',
}).addTo(map);

var customExports = mapknitter.customExportAction();
var imgGroup = L.distortableCollection({
actions: [customExports],
}).addTo(map);
var imgGroup = L.distortableCollection({}).addTo(map);

imgGroup.addLayer(img);

Expand Down Expand Up @@ -617,9 +617,81 @@ MapKnitter.Map = MapKnitter.Class.extend({
setupCollection: function() {

map._imgGroup = L.distortableCollection({
editable: !mapknitter.readOnly
editable: !mapknitter.readOnly,
exportOpts: {
// exportUrl: 'http://34.74.118.242/api/v2/export/', // to swap to JS exporter
// exportStartUrl: 'http://34.74.118.242/api/v2/export/', // to swap to JS exporter
fetchStatusUrl: fetchStatusUrl
}
}).addTo(map);

// customize the function that starts up the export
function fetchStatusUrl(opts) {
console.log('fetch status json', opts);

var scale = 0;
opts.collection.forEach(function(img) {
scale += img.cm_per_pixel;
});
// average of scales of each image
scale = parseInt(scale/opts.collection.length);

$.ajax({
url: 'https://export.mapknitter.org/export',
crossDomain: true,
type: 'POST',
data: {
collection: JSON.stringify(opts.collection),
scale: prompt("Choose a scale in 'centimeters per pixel' (where a smaller 50cm pixel is higher resolution - comparable to Google Maps - or a larger 200cm pixel is lower resolution):", scale) || opts.scale,
upload: true,
},
success: handleStatusResponse
});
// show exports
$('.export-tab').click();
$('.exports-tab').click();
window.location.hash = "#cloud-exports";
}

// receives the URL of status.json, and starts running the updater to repeatedly fetch from status.json;
// this may be overridden to integrate with any UI
function handleStatusResponse(status_url, opts) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See how here, I am now passing the opts object through each function, so that the functions don't need to have access to internal scope from the Collection; I had hoped that at the end of this function the opts.resolve() would work.

alert("Export has begun: leave this window open to be notified of completion. Due to a known issue, please refresh the page if you'd like to start another."); // https://github.com/publiclab/Leaflet.DistortableImage/issues/522
// this is for the JS exporter:
// var statusUrl = data.split('please visit, ')[1];

// save the location of the status URL
$.ajax({
url: "/export",
type: 'POST',
data: {
status_url: 'https://export.mapknitter.org' + status_url,
map_id: mapknitter.map_id
}
}).done(function (data) {
console.log('saved status.json URL to MapKnitter', data);
});

// repeatedly fetch the status.json
var updateInterval = setInterval(function intervalUpdater() {
$.ajax('https://export.mapknitter.org/' + status_url + '?' + Date.now(), { // bust cache with timestamp
type: 'GET',
crossDomain: true,
}).done(function(data) {
// update the progress bar or spinner
// opts.updater(data);
data = JSON.parse(data);
console.log(data, data.status, data.jpg);
if (data && data.status == "complete") {
alert("Export completed at " + data.cm_per_pixel + " cm/px. JPG available at https://mapknitter-exports-warps.storage.googleapis.com/" + data.jpg.split('public/warps')[1] + " -- Please refresh page to view completed exports.");
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here I set the alert to show the JPG and direct towards a reload. then cleared the interval. Wooo!

clearInterval(updateInterval);
}
});
}, 3000); // frequency of updating

opts.resolve(); // stop the spinner
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But maybe we're not passing resolve into the opts object? I had expected this to work but opts doesn't seem to contain the promise @VladimirMikulic

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe if I do this, it'll work? @VladimirMikulic are you able to review this: publiclab/Leaflet.DistortableImage#528

Thank you!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, fixed and merged!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jywarren sorry for not being available earlier. I am glad that you've managed to fix it 👍

}

var sidebar = document.querySelector('body > div.sidebar');

if (!mapknitter.readOnly) {
Expand Down Expand Up @@ -668,4 +740,4 @@ MapKnitter.Map = MapKnitter.Class.extend({
}

});

//})();
7 changes: 7 additions & 0 deletions app/assets/stylesheets/style.scss
Original file line number Diff line number Diff line change
Expand Up @@ -317,3 +317,10 @@ label small {
width: 100%
}
}

#open_id_form button {
width: 70px;
background-color: #ffffff;
border: none;
margin-right: 0.25rem;
}
7 changes: 6 additions & 1 deletion app/controllers/export_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,12 @@ def create

# mySQL2 error ActiveRecord::StatementInvalid (Mysql2::Error: Field 'bands_string' doesn't have a default value: INSERT INTO `exports` (`export_url`, `created_at`, `updated_at`) VALUES ('//export.mapknitter.org/id/1562102960/status.json', '2019-07-02 21:29:20', '2019-07-02 21:29:20')):
# so adding a default value for now. I think this column will be deprecated?
export = Export.create!(export_url: params[:status_url], bands_string: 'default bands_string')
export = Export.create!(
export_url: params[:status_url],
user_id: current_user&.id || 0,
map_id: params[:map_id],
bands_string: 'default bands_string'
)
render json: export.to_json
end

Expand Down
8 changes: 0 additions & 8 deletions app/views/layouts/_login_form.html.erb
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,3 @@
<% end %>
</p>
</form>
<style>
#open_id_form button{
width: 70px;
background-color: #ffffff;
border: none;
margin-right: 0.25rem;
}
</style>
11 changes: 7 additions & 4 deletions app/views/layouts/knitter2.html.erb
Original file line number Diff line number Diff line change
Expand Up @@ -98,14 +98,16 @@
<a class="nav-link" href="#map-images-container" role="tab" data-toggle="tab">Images</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#map-editing-container" role="tab" data-toggle="tab">Annotate</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#map-exports-container" role="tab" data-toggle="tab">Export</a>
<a class="nav-link export-tab" href="#map-exports-container" role="tab" data-toggle="tab">Export</a>
</li>
<li class="nav-item d-none d-sm-block">
<a class="nav-link" target="_blank" href="https://publiclab.org/chat" role="tab" style="border: none;">Chat</a>
</li>
<!--
<li class="nav-item">
<a class="nav-link" href="#map-editing-container" role="tab" data-toggle="tab">Annotate</a>
</li>
-->
</ul>

<!-- Tab panes -->
Expand Down Expand Up @@ -191,6 +193,7 @@
readOnly: <%= @embed == true || @annotations == true %>,
logged_in: <%= logged_in? == true %>,
anonymous: <%= @map.anonymous? == true %>,
map_id: <%= @map.id || 0 %>,
warpablesUrl: "<%= url_for([@map, :warpables])+'.json' unless @map.warpables.empty? %>"
});
<% if @map.warpables.empty? && params[:action] == 'edit' && @map.display_welcome %>
Expand Down
32 changes: 32 additions & 0 deletions app/views/maps/_cloud_exports.html.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
<p><a href="https://publiclab.org/wiki/mapknitter-cloud-exporter">Using the Cloud Exporter &raquo;</a></p>
<div class="mapknitter-cloud-exports" id="cloud-exports">
<% exports.order(created_at: :desc).each_with_index do |export, i| %>
<p class="export mapknitter-cloud-export" style="font-size:11px;">
<b>Export <%= i %>:</b>
(<a href="<%= export.export_url %>">status</a>)
</p>
<% end %>
</div>
<script>
(function() {
$('.mapknitter-cloud-exports .mapknitter-cloud-export').each(function collectExportUrls() {
var export_el = $(this);
var export_url = export_el.find('a').attr('href');
$.ajax(export_url).done(function displayExportStatus(response) {
response = JSON.parse(response);
console.log(response, response.status);

if (response.status) export_el.append(' <i class="status">' + response.status + '</i>');
if (response.jpg) export_el.append(' <span class="file"><a href="https://mapknitter-exports-warps.storage.googleapis.com/' + response.jpg.split('public/warps/')[1] + '">JPG</span>');
if (response.geotiff) export_el.append(' <span class="file"><a href="https://mapknitter-exports-warps.storage.googleapis.com/' + response.geotiff.split('public/warps/')[1] + '">GeoTiff</span>');

export_el.append('<br />');

if (response.height && response.width) export_el.append(' <span class="meta">' + response.height + 'x' + response.width + '</span>');
if (response.size) export_el.append(' <span class="meta">' + (response.size.split('B')[0]/1000000) + 'MB</span>');
if (response.zip) export_el.append(' <span class="file"><a href="https://mapknitter-exports-warps.storage.googleapis.com/' + response.zip.split('public/tms/')[1] + '">TMS (zip)</span>');
if (response.start_time) export_el.append(' <i class="start_time">' + response.start_time + '</i>');
});
})
})()
</script>
19 changes: 17 additions & 2 deletions app/views/maps/_sidebar_exports.html.erb
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,13 @@

<ul class="nav nav-tabs">
<li class="nav-item">
<a class="nav-link active" href="#map-export-subtab" role="tab" data-toggle="tab">Export</a>
<a class="nav-link active" href="#map-export-subtab" role="tab" data-toggle="tab">Start export</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#map-export-options-subtab" role="tab" data-toggle="tab">Advanced option</a>
<a class="nav-link exports-tab" href="#map-exports-subtab" role="tab" data-toggle="tab">Exports</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#map-export-options-subtab" role="tab" data-toggle="tab">Advanced</a>
</li>
</ul>

Expand All @@ -42,6 +45,12 @@
<% end %>

<% if logged_in? || @map.anonymous? %>

<div class="new-exporter">
<p>MapKnitter now has a new beta Cloud Exporter. To use it, select multiple images (with the shift key, or by long-pressing) and click the Download <code><i class='fa fa-download'></i></code> button in the upper left of the map.</p>
<p>Exporting may take many minutes; keep the window open as it runs if you'd like to be notified when it completes. <a href="https://publiclab.org/wiki/mapknitter-cloud-exporter">Using the Cloud Exporter &raquo;</a></p>
</div>

<p>
<button id="begin-export" class="btn btn-lg btn-primary"<% if @map.export && @map.exporting? %> style="display:none;"<% end %>>Start export</button>
<button <% unless @map.export && @map.exporting? %>style="display:none;"<% end %> id="cancel-export" class="btn btn-lg btn-outline-secondary"><i class="fas fa-times"></i> Cancel export</button>
Expand Down Expand Up @@ -69,6 +78,12 @@
</p>
</div>

<div role="tabpanel" class="tab-pane active" id="map-exports-subtab">
<p><b>Exports generated by the Cloud Exporter</b></p>
<hr />
<%= render partial: 'maps/cloud_exports', locals: { exports: @map.exports }%>
</div>

<div class="tab-pane" id="map-export-options-subtab">
<p>Tweak your export resolution; too high a resolution may stall the export (if the resulting file is too big!), while too low a resolution causes lots of processing to resize the images.</p>

Expand Down
6 changes: 6 additions & 0 deletions app/views/maps/show.html.erb
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,12 @@
<%= render partial: 'front_ui/maps', locals: { maps: @maps } %>
</div>

<div class="all-maps light-blue">
<br>
<h3 class="text-center"> Exports </h3>
<%= render partial: 'maps/cloud_exports', locals: { exports: @map.exports }%>
</div>

<script>
var map
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test/system/images_test.rb failure of map._layers doesn't make sense unless this line was interrupted. I'm restarting Travis just to check.

https://travis-ci.org/publiclab/mapknitter/jobs/642090297

(function(){
Expand Down
4 changes: 2 additions & 2 deletions config/environments/production.rb
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@
config.public_file_server.enabled = ENV['RAILS_SERVE_STATIC_FILES'].present?

# Compress JavaScripts and CSS.
require 'uglifier'
config.assets.js_compressor = Uglifier.new(harmony: true)
config.assets.js_compressor = Uglifier.new(harmony: true, compress: { unused: false })
#config.assets.js_compressor = :uglifier
# config.assets.css_compressor = :sass

# Do not fallback to assets pipeline if a precompiled asset is missed.
Expand Down
1 change: 1 addition & 0 deletions config/initializers/assets.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
# Add additional assets to the asset load path.
# Rails.application.config.assets.paths << Emoji.images_path
# Add Yarn node_modules folder to the asset load path.
config.assets.paths << Rails.root.join('public/lib')
config.assets.paths << Rails.root.join('node_modules')

config.assets.precompile << /\.(?:svg|eot|woff|ttf)\z/
Expand Down
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@
"junction": "theleagueof/junction#*",
"leaflet": "Leaflet/Leaflet#^1.0.0",
"leaflet-dist": "vperron/leaflet-dist#0.7.x",
"leaflet-distortableimage": "0.11.6",
"leaflet-draw": "Leaflet/Leaflet.draw#0.2.4",
"leaflet-distortableimage": "0.8.7",
"leaflet-easybutton": "^2.4.0",
"leaflet-environmental-layers": "^2.1.9",
"leaflet-fullhash": "^1.0.0",
Expand All @@ -36,7 +36,7 @@
"leaflet-spin": "^1.1.0",
"modalbox": "okonet/modalbox#*",
"ol2": "openlayers/ol2#release-2.13.1",
"popper": "^1.0.1",
"popper.js": "^1.16.0",
"prototypejs-bower": "plynchnlm/prototypejs-bower#1.7.3",
"sparklines": "mariusGundersen/sparkline#~1.2.0",
"spin.js": "fgnass/spin.js#^2.3.1",
Expand Down
Loading