Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Located Triples #1351

Draft
wants to merge 152 commits into
base: master
Choose a base branch
from
Draft

Located Triples #1351

wants to merge 152 commits into from

Conversation

Qup42
Copy link
Member

@Qup42 Qup42 commented May 19, 2024

SPARQL 1.1 Update. The Graph Management Operations are dependent on #1337 and not supported yet.

Status

It should somewhat work. You can try out some queries like

  1. Insert triples
INSERT DATA {
  <http://wallscope.co.uk/resource/olympics/athlete/MeganOlwenDevenishTaylorMandevilleEllis> <foo> <baz>
}

or

PREFIX foaf: <http://xmlns.com/foaf/0.1/>
INSERT {
  ?a <foo> <baz>
} WHERE {
  ?a foaf:gender <http://wallscope.co.uk/resource/olympics/gender/F> .
}
  1. Retrieve triples
SELECT * WHERE {
  <http://wallscope.co.uk/resource/olympics/athlete/MeganOlwenDevenishTaylorMandevilleEllis> ?b ?c 
}

or

SELECT * WHERE {
  ?a <foo> <baz>
}

TODOs:

  • Split up into smaller chunks for PRs
    • LocatedTriple
    • DeltaTriple
    • integration
  • Unit Tests
    • CompressedRelationReader -> together with integration PR
    • DeltaTriples -> WIP
  • Better feedback of processing time for to guide future (optimization) decisions

Hannah Bast and others added 22 commits June 9, 2023 16:47
This is the first part of a series of PRs split of from the large
proof-of-concept PR ad-freiburg#916,
which realizes SPARQL 1.1 Update
This PR is from June 2023. Since then, quite a bit of code that is
relevant for this PR has been refactored. In particular: the
`Permutation` class, the `CompressedRelationWriter` and the index
building functions in `IndexImpl`. Mostly, this refactoring has made the
code in this PR simpler. There is still some akwardness in
`LocatedTriplesTest.cpp` because we want to build an index from `Id`
triples there (and not from Turtle input).

Now evertyhing compiles and runs through again. Various tests in
`LocatedTriplesTest` fail, that's what I will look at next.
This makes the actual changes hard to spot
Both cases would have done a full scan anyway, so we can do that directly without retrieving the relation metadata.
# Conflicts:
#	test/ValueIdTest.cpp
@Qup42
Copy link
Member Author

Qup42 commented May 20, 2024

Some data:

  • Built with Release CMake Profile against cfc581e
  • Executed on my Laptop (AMD Ryzen 7840U, 32GB RAM, SSD)
  • Olympics dataset
    • ~400k and ~8M triples are to be located, once each for deletion and insertion
    • times are averaged over all permutations

~400k

IdTable, random-access IdTable, pre-sorting std::vector, random-access std::vector pre-sorting
Result materialisation 70ms 70ms 70ms 70ms
Sorting N/A 60ms-160ms N/A 20ms
Location 15ms-20ms 10ms-20ms 15ms-20ms 10ms-20ms

~8M

IdTable, random-access IdTable, pre-sorting std::vector, random-access std::vector pre-sorting
Result materialisation 1330ms 1330ms 1330ms 1330ms
Sorting N/A 1500ms-4000ms N/A 460ms-510ms
Location 330ms-450ms 240ms-360ms 330ms-460ms 220ms-350ms

@Qup42 Qup42 mentioned this pull request May 22, 2024
Qup42 added 29 commits July 4, 2024 17:03
# Conflicts:
#	test/util/IdTestHelpers.h
# Conflicts:
#	src/global/IdTriple.h
#	src/index/IndexFormatVersion.h
#	src/index/LocatedTriples.cpp
#	src/index/LocatedTriples.h
#	test/CMakeLists.txt
#	test/IdTripleTest.cpp
#	test/LocatedTriplesTest.cpp
# Conflicts:
#	src/engine/sparqlExpressions/SparqlExpressionValueGetters.cpp
Copy link

sonarcloud bot commented Jul 23, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant