Import of graphic, layout and image objects #2124

dl3sdo · 2023-01-10T12:53:32Z

Graphic, layout and image objects are stored in .ocd without dedicated symbols.
While graphic objects re-use existing symbol colors, image and layout objects store the object color inside the object.
Mapper imports these special objects by defining auxiliary symbols and colors.
Layout image objects, i.e.,objects that reference externally stored images cannot yet be imported, however, a warning is now issued instead of silently discarding them.

Closes #959
Closes #1231

OCD format recognizes symbol-less map objects that, however, render in map colors. They are called "graphic objects". We create synthetic symbols for graphic objects on import. All the Mapper symbols have the same name and number reflecting the anonymous nature of the source graphic objects. Closes OpenOrienteeringGH-959.

There is a flag in the .ocd file to hide all graphics objects which was not yet evaluated. The status of the flag is now read and applied to all imported graphic objects.

Layout objects (line, area, text) can now be imported. Significant code parts were taken over from Libor's commits. Layout objects' symbols are hidden depending on the 'l' flag in si_ViewPar. The 'l' flag in si_DisplayPar is apparently not used for hiding layouts objects, but both flags are evaluated in parallel. Todo: for text objects the import of the format (font, size, alignment, bold, italic) needs to be added.

Refactor importDisplayPar() by using OcdParameterStreamReader.

There is no need to cache the dynamically created graphic and layout symbols. Instead create a temporary symbol and then scan the list of symbols whether an equal symbol is already present and thus can be used instead.

Text layout objects contain a limited set of formatting options, e.g., font type, font size, horizontal alignment, italic and bold type. These options are stored after the text string in the same format as for Parameter Strings.

In addition to 'addWarning()' 'addWarningOnce()' only adds a warning to the list of warnings if the warning is not yet in the list.

Image objects (as well as layout objects) define their color directly in the object definition. With this commit the import of layout, image and graphic objects is now done within a single function. A single warning is issued that these kind of objects are imported by means of auxiliary symbols and can't be exported in their original format. Closes OpenOrienteering#1231

Layout objects can be hidden individually. This property is set within a parameter string (type 27). If the parameter string defines an image layout object, a warning is issue that this object type cannot be imported.

Setting of cap and join styles was done in the importLineSymbolBase() function and was replicated by this PR for importing special objects. In addition setupLineSymbolFraming() was setting the cap and join styles for framing lines (although only checking for a subset of cap and join styles). This commit moves the setting to a separate function.

dl3sdo · 2023-01-10T20:10:10Z

Test data:
LayoutGraphicImageObjects7_partiallyHiddenWithImage.zip

Libor: thank you for reviewing

Image layout objects are stored in parameter strings very similar to background images (i.e., templates), whereas the image file itself is stored outside the .ocd file. This commit imports image layout objects as templates (and informs the user about it) after the original templates have been imported and thus adds them to the end of the list of templates.

lpechacek

Thanks, Matthias for the thorough implementation! I like the functionality a lot.

I have a few questions about the code though and some request about the patch set organization. I know how painful patch set rewrites tend to be but in my opinion some of the later patches ("OcdFileImport: Apply OcdParameterStreamReader", "OcdFileImport: Import of image objects", "OcdFileImport: Set cap and join styles in separate function") should be integrated into the earlier ones. I have indicated that in my comments as well.

The patch "ImportExport: Avoid duplicate warnings" should be renamed to make it clear that it introduces a new function rather than changing existing code. I would also suggest merging the change with the patch that uses the function for the first time.

The patch set adjustment should lead to a lower number of patches and a better readable change log. I'm looking forward to the next patch set version!

lpechacek · 2023-03-06T14:18:03Z

src/fileformats/ocd_file_import.cpp

@@ -107,6 +107,7 @@ OcdFileImport::OcdImportedPathObject::~OcdImportedPathObject() = default;
 OcdFileImport::OcdFileImport(const QString& path, Map* map, MapView* view)
 : Importer { path, map, view }
 , custom_8bit_encoding { codecFromSettings() }
+ , graphic_objects_hidden(false)


Please use uniform initialization (graphic_objects_hidden { false }) for the member. Not that my code would be perfect in this aspect but the uniform initialization better fits the existing code here in ocd_file_import.cpp and also ocd_file_export.cpp.

My comment also applies to the other class members added further down the patch set.

lpechacek · 2023-03-06T14:20:08Z

src/fileformats/ocd_file_import.cpp

+ int i = param_string.indexOf(QLatin1Char('\t'), 0);
+ ; // skip first word for this entry type


Just a nit ... or in other words, why repeat old sins. Use auto for the variable. Fix grammar in the comment ("skip the first ...").

Well, and after reading the rest of the patch set, I can see that this C&P goes away anyway. Please use the OcdParameterStreamReader right from the beginning. I.e. squash "OcdFileImport: Apply OcdParameterStreamReader " into this patch.

lpechacek · 2023-03-06T17:26:41Z

src/fileformats/ocd_file_import.cpp

+ for (int i = 0; i < num_symbols; ++i)
+ {
+ if (symbol->equals(map->getSymbol(i)))
+ {
+ delete symbol;
+ return map->getSymbol(i);
+ }
+ }


Creating a symbol object and for every imported graphic object and then scanning the symbol list and potentially deleting the symbol object feels wasteful to me. Maybe I'm old school but maybe not that much when my views are reflected also in C++ Guidelines.

What is the reason for dropping the symbol hash structure, please?

The hash structure was not applicable (easily) for text symbols: the hash value was previously created for line and area symbols from the type (line/area), color and width and thus fitted good into a 64 bit type. However, for text symbols there are additional properties: font size, bold/italic, font family. And for line symbols the cap and join styles had not been considered before.
There is a small runtime optimization possible that I did not apply. Instead of searching through the whole list of symbols it's sufficient to remember the index of the first auxiliary symbol and then start searching from there on.

lpechacek · 2023-03-06T20:17:37Z

src/fileformats/ocd_file_import.cpp

 {
- symbol = symbol_index[ocd_object.symbol];
+ case -2: // graphic object


Please inroduce the enums for object types (now in patch "OcdFileImport: Import of image objects") in this patch and use the enum right from the beginning.

Don't worry about rewriting my patch. Take the change ownership and name me as a co-author.

src/fileformats/ocd_file_import.cpp

lpechacek · 2023-03-06T21:02:15Z

src/fileformats/ocd_file_import.cpp

+ else if (ocd_object.symbol == Ocd::ImageObject)
+ addWarningOnce(tr("Importing image objects using auxiliary symbols. Export as image objects is not possible."));
+ else
+ addWarningOnce(tr("Importing graphic objects using auxiliary symbols. Export as graphic objects is not possible."));


I very much like these warnings and thought about them some time ago. Can you please look into integrating the warning messages into the earlier patches that implement the respective functionality?

lpechacek · 2023-03-06T21:06:09Z

src/fileformats/ocd_file_import.h

 /// The protection and visibility state of image objects (bitfield: 1 = protected, 2 = hidden)
 int image_objects_displaymode;
+
+ /// counting objects during import
+ int object_index;


I'm not completely sure about the purpose and naming of this member. Can you make its purpose more obvious, please?

lpechacek · 2023-03-06T21:07:30Z

src/fileformats/ocd_file_import.cpp

@@ -1302,22 +1302,13 @@ Symbol* OcdFileImport::importLineSymbol(const S& ocd_symbol)
 return combined_line;
 }

-void OcdFileImport::importLineSymbolBase(OcdImportedLineSymbol* symbol, const Ocd::LineSymbolCommonV8& attributes)
+bool OcdFileImport::setupLineStyle(OcdImportedLineSymbol* symbol, const quint16 line_style)


Another welcome deduplication! Could it go earlier in the patch set?

lpechacek · 2023-03-06T21:12:08Z

src/fileformats/ocd_file_import.cpp

@@ -1131,6 +1140,39 @@ void OcdFileImport::importLayoutObjects(const QString& param_string)
 if (ok)
 object_number = i_value;
 break;
+ case 'x':


I know that these parameters are shared with template image import. That said, is there a chance of introducing a shared function for handling the x, y, a, b, u and v parameters?

lpechacek · 2023-03-06T21:16:57Z

src/fileformats/ocd_file_import.cpp

+ y_value = f_value;
+ break;
+ case 'a':
+ case 'b':


As a side note. As to my understanding, a is in principle the angle of image x axis and b is the angle of the y axis in the map grid. The TODO should not be addressed in this patch set but if someone wants to look into it, here's the clue.

Of course, it will be easier for the future coder to adjust just one place in the code rather than fixing templates and image objects separately.

I started having a look on that topic and already have a test file. Like you I decided that this fix is subject of another PR and thus deferred it.

I also spent some time with the topic. I can load and save template transformation in .ocd files (see the attached archive) but the change breaks all the other template handling in Mapper.
template-ab-params.zip

The thing is that TemplateTransform::template_shear is currently an opaque number that somehow embodies the actual template shear, along with other values like scaling and rotation. That said, the patch that introduces this class member needs a thorough review.

lpechacek · 2023-03-08T12:05:15Z

@dl3sdo, just to let you know, I've just spotted and .ocd with embedded layout image. It uses an undocumented code 'F'. Abbreviated string dump from Mapper follows.

The .ocd can be downloaded from https://drive.google.com/drive/folders/1QR2jLfoq8pEss5z-eLqIaFI0u3d6uf1N?usp=sharing . The archive contains 'ocad12/Grafika výuková mapa_A4výška_ocad12.ocd' that has the layout thingie. The file is Base64 encoded.

You may want to test your layout object code with this file to see how it behaves.

 27 	mapový klíč.jpg	r1	s1	F/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAMCAgMCAgMDAwMEAwMEBQgFBQQEBQoHBwYIDAoMDAsK
CwsNDhIQDQ4RDgsLEBYQERMUFRUVDA8XGBYUGBIUFRT/2wBDAQMEBAUEBQkFBQkUDQsNFBQUFBQU
FBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBT/wgARCANbBHoDASIA
AhEBAxEB/8QAHQABAAIDAQEBAQAAAAAAAAAAAAYHBAUIAwECCf/EABsBAQEAAwEBAQAAAAAAAAAA
AAABAgQFAwYH/9oADAMBAAIQAxAAAAHqkAAAAAAAAAAAAAAAAAAAABg5J6gAAANONw88E2QAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABx4dhqit0KBjS9RnLqdRKKvUOLpC
....
73gqnGgTMY1lUdZwRBjRZuNIXUpQjs9g3oE8EjyoQ5ogt4ArppwG/IrCNy3m5SCCDdy6CTZKrVr/
ACAQYwJDQFARVpaAIpQQ6DhoLeuh6uhlDAR041kMdxwTBjRZuNIFz29veTL1cL3eIW3nTwocYS0N
7p7+Gg+6H1FSbwYhbrsG0JSKmCBgQWEhWWEIHgpTL7fkZEBiDCAznkuGFtRghRRmLwVKeaOgMJAG
mpRcPlDbtlRSsarA6OuIayuB2h0IzeKyTfqscKoAVZ6cGfjCy6UN3pBvdMmySpmYElKWB4KrSqDR
XEgQgSBT5ZjF2s0Nnpn+KCwdakQOAU2IQkBwx7S9cAp1hJB3q/51yorneJM06VLWvxm2KhwArMJW
kTo/NdIIW2QLACmIPjnaCFtkCUQhqr5//ih//9k=
	u0.039375	v0.039375	x176.490000	y-237.220000

The .ocd v2018 variant also contains the embedded image.

Layout raster image objects can also be stored as embedded images directly in the .ocd file. This commit allows to save the embedded image locally in order to be added as a template to the map.

dl3sdo · 2023-03-17T21:52:35Z

@lpechacek: Before addressing your proposed changes I wanted to add the import of embedded images.

lpechacek and others added 9 commits January 6, 2023 16:25

OcdFileImport: Hide imported graphics objects

47bf466

There is a flag in the .ocd file to hide all graphics objects which was not yet evaluated. The status of the flag is now read and applied to all imported graphic objects.

OcdFileImport: Apply OcdParameterStreamReader

244226a

Refactor importDisplayPar() by using OcdParameterStreamReader.

OcdFileImport: Complete import of text layout objects

0c18743

Text layout objects contain a limited set of formatting options, e.g., font type, font size, horizontal alignment, italic and bold type. These options are stored after the text string in the same format as for Parameter Strings.

ImportExport: Avoid duplicate warnings

5d86a92

In addition to 'addWarning()' 'addWarningOnce()' only adds a warning to the list of warnings if the warning is not yet in the list.

OcdFileImport: Hide layout objects individually

62f1e96

Layout objects can be hidden individually. This property is set within a parameter string (type 27). If the parameter string defines an image layout object, a warning is issue that this object type cannot be imported.

dl3sdo marked this pull request as draft January 10, 2023 18:45

lpechacek self-requested a review January 10, 2023 19:30

lpechacek requested changes Mar 6, 2023

View reviewed changes

OcdFileImport: Import embedded images

71f821c

Layout raster image objects can also be stored as embedded images directly in the .ocd file. This commit allows to save the embedded image locally in order to be added as a template to the map.

dl3sdo mentioned this pull request Jul 29, 2024

Ignore virtual gaps in line objects #2266

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Import of graphic, layout and image objects #2124

Import of graphic, layout and image objects #2124

dl3sdo commented Jan 10, 2023

dl3sdo commented Jan 10, 2023

lpechacek left a comment

lpechacek Mar 6, 2023

lpechacek Mar 6, 2023

lpechacek Mar 6, 2023

lpechacek Mar 6, 2023

dl3sdo Mar 7, 2023

lpechacek Mar 6, 2023

lpechacek Mar 6, 2023

lpechacek Mar 6, 2023

lpechacek Mar 6, 2023

lpechacek Mar 6, 2023

lpechacek Mar 6, 2023

dl3sdo Mar 7, 2023

lpechacek Mar 8, 2023

lpechacek commented Mar 8, 2023

dl3sdo commented Mar 17, 2023

		int i = param_string.indexOf(QLatin1Char('\t'), 0);
		; // skip first word for this entry type

Import of graphic, layout and image objects #2124

Are you sure you want to change the base?

Import of graphic, layout and image objects #2124

Conversation

dl3sdo commented Jan 10, 2023

dl3sdo commented Jan 10, 2023

lpechacek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lpechacek commented Mar 8, 2023

dl3sdo commented Mar 17, 2023