Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: validate local and external references #26

Closed
wants to merge 32 commits into from
Closed
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
c0ba854
feat: validate local and external references
probablyArth Mar 14, 2024
20eca6e
use getKeywordName for fetching keyword name for reference keyword
probablyArth Mar 17, 2024
5ce414d
use the existing JsoncInstance class
probablyArth Mar 17, 2024
a033849
add contextDialectUri to toplevel
probablyArth Mar 17, 2024
5cf6eed
Merge remote-tracking branch 'upstream/main' into feat-validate-refer…
probablyArth Mar 17, 2024
c913db3
lint
probablyArth Mar 19, 2024
3873847
Merge branch 'main' into feat-validate-references/#7
probablyArth Mar 19, 2024
d96ee47
temp commit
probablyArth Mar 27, 2024
6b09cc2
handle external reference JSON pointer
probablyArth Mar 28, 2024
899d3c3
Merge remote-tracking branch 'upstream/main' into feat-validate-refer…
probablyArth Mar 28, 2024
84c895c
merge
probablyArth Mar 28, 2024
a0abcd8
modify references logic according to the latest merge
probablyArth Mar 28, 2024
8700c8f
feat: anchor fragment
probablyArth Mar 28, 2024
aaafff9
fix relative ref
probablyArth Mar 28, 2024
def18f8
extract fetching file logic
probablyArth Mar 28, 2024
29ce34d
modify jsonc-instance
probablyArth Mar 28, 2024
69e3ba7
external references with absolute uri
probablyArth Mar 28, 2024
0159120
write cleaner statements
probablyArth Mar 28, 2024
21a7a53
fix absolute url regex
probablyArth Mar 29, 2024
cfb727e
add catch for linux on fs.watch
probablyArth Mar 29, 2024
4e02c20
remove catch for fs.watch
probablyArth Mar 29, 2024
1b44b78
resolve conflicts
probablyArth Mar 30, 2024
33afa77
set recursive flag to true for watch in workspace
probablyArth Apr 2, 2024
3dea88c
feat: identifiers, inactiveDocuments and referencesStore
probablyArth Apr 5, 2024
e2800ab
Merge remote-tracking branch 'upstream/main' into feat-validate-refer…
probablyArth Apr 5, 2024
1d97c25
pass value of ref instead of JsoncInstance
probablyArth Apr 5, 2024
d10b124
feat: validate references
probablyArth Apr 6, 2024
fc6f164
refactor: delete identifiers store, refactor references logic
probablyArth Apr 24, 2024
caaa0a1
remove redundant creation of JsoncInstance
probablyArth May 11, 2024
e85afb2
remove inactiveDocumentStore
probablyArth May 11, 2024
5b7a676
nothing should be exported from the server
probablyArth May 11, 2024
96a196d
move isSchema back to server.js
probablyArth May 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions language-server/src/documents.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
import { readFile } from "fs/promises";
import { fileURLToPath } from "url";
import { TextDocument } from "vscode-languageserver-textdocument";

/**
* documentStore for inactive documents
*
* `uri` => `TextDocument`
* @type {Map<string, TextDocument}
*/
const inactiveDocumentStore = new Map();

/**
* @param {import("vscode-languageserver").TextDocuments<TextDocument>} documents
* @param {string} uri
* @returns {Promise<TextDocument>}
*/
export const fetchDocument = async (documents, uri) => {
if (inactiveDocumentStore.has(uri)) {
return inactiveDocumentStore.get(uri);
}

let textDocument = documents.get(uri);

if (!textDocument) {
const instanceJson = await readFile(fileURLToPath(uri), "utf8");
textDocument = TextDocument.create(uri, "json", -1, instanceJson);
inactiveDocumentStore.set(uri, textDocument);
}
return textDocument;
};

/**
* @param {string} uri
*/
export const deleteFromInactiveDocumentStore = (uri) => {
return inactiveDocumentStore.delete(uri);
};
63 changes: 63 additions & 0 deletions language-server/src/identifiers.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
import { keywordNameFor } from "./json-schema.js";

/**
* `$id` -> `document uri`
* @type {Map<string, string>}
*/
const identifiers = new Map();

/**
* `uri` -> `$id -> instance`
* @type {Map<string, Map<string, import("./jsonc-instance").JsoncInstance>}
*/
const embeddedSchemaIdentifiers = new Map();

export const deleteIdentifiersForDocument = (uri) => {
embeddedSchemaIdentifiers.delete(uri);
identifiers.forEach((docUri) => {
if (docUri === uri) {
identifiers.delete(docUri);
return;
}
});
};

/**
* @param {import("./jsonc-instance").JsoncInstance} instance
* @param {string} dialectUri
*/
export const addIdentifierForInstance = (instance, dialectUri) => {
const idKeywordUri = "https://json-schema.org/keyword/id";
const $id = instance.get(`#/${keywordNameFor(idKeywordUri, dialectUri)}`).node?.value;

const { textDocument, node } = instance;

if (node && node.offset !== 0) {
if (!embeddedSchemaIdentifiers.has(textDocument.uri)) {
embeddedSchemaIdentifiers.set(textDocument.uri, new Map());
}
embeddedSchemaIdentifiers.get(textDocument.uri).set($id, instance);
return;
}

if ($id) {
identifiers.set($id, textDocument.uri);
} else {
identifiers.set(textDocument.uri, textDocument.uri);
}
};

/**
* @param {string} $id
*/
export const getIdentifier = ($id) => {
return identifiers.get($id);
};

/**
* @param {string} uri
* @param {string} $id
*/
export const getEmbeddedIdentifer = (uri, $id) => {
return embeddedSchemaIdentifiers.get(uri)?.get($id);
};
8 changes: 7 additions & 1 deletion language-server/src/json-schema.js
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,13 @@ const getEmbeddedDialectUri = (schemaInstance, contextDialectUri) => {
}
};

const keywordNameFor = (keywordUri, dialectUri) => {
/**
*
* @param {string} keywordUri
* @param {string} dialectUri
* @returns {string | undefined}
*/
export const keywordNameFor = (keywordUri, dialectUri) => {
try {
return getKeywordName(dialectUri, keywordUri);
} catch (error) {
Expand Down
11 changes: 11 additions & 0 deletions language-server/src/jsonc-instance.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,13 @@ import { toAbsoluteUri, uriFragment } from "./util.js";


export class JsoncInstance {
/**
* @param {import("vscode-languageserver-textdocument").TextDocument} textDocument
* @param {import("jsonc-parser").Node | undefined} root
* @param {import("jsonc-parser").Node | undefined} node
* @param {string} pointer
* @param {*} annotations
*/
constructor(textDocument, root, node, pointer, annotations) {
this.textDocument = textDocument;
this.root = root;
Expand All @@ -14,6 +21,10 @@ export class JsoncInstance {
this.annotations = annotations;
}

/**
* @param {import("vscode-languageserver-textdocument").TextDocument} textDocument
* @returns {JsoncInstance}
*/
static fromTextDocument(textDocument) {
const json = textDocument.getText();
const root = parseTree(json, [], {
Expand Down
174 changes: 174 additions & 0 deletions language-server/src/references.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
import { fetchDocument } from "./documents.js";
import { getEmbeddedIdentifer, getIdentifier } from "./identifiers.js";
import { keywordNameFor } from "./json-schema.js";
import { JsoncInstance } from "./jsonc-instance.js";
import { getSemanticTokens } from "./semantic-tokens.js";
import { getSchemaResources, workspaceUri } from "./server.js";
probablyArth marked this conversation as resolved.
Show resolved Hide resolved
import { isAbsoluteUrl, isAnchor } from "./util.js";
import { dirname, join } from "node:path";
import { pathToFileURL, resolve } from "node:url";

/**
* `documentUri` -> `ref -> instance`
* @type {Map<string, Map<string, import("./jsonc-instance.js").JsoncInstance>}
*/
export const references = new Map();

/**
* @param {string} uri
*/
export const getReferenceForDocument = (uri) => {
return references.get(uri);
};

export const deleteReferencesForDocument = (uri) => {
references.delete(uri);
};

/**
* @param {import("./jsonc-instance.js").JsoncInstance} keywordInstance
* @param {string} dialectUri
*/
export const addReference = (keywordInstance, dialectUri) => {
const referenceKeywordIds = [
"https://json-schema.org/keyword/ref",
"https://json-schema.org/keyword/draft-04/ref"
];
const referenceKeywordNames = referenceKeywordIds.map((keywordId) => keywordNameFor(keywordId, dialectUri));
if (!referenceKeywordNames.includes(keywordInstance.value()) || keywordInstance?.node?.parent?.children?.length < 2) {
return;
}
const { pointer, textDocument, node } = keywordInstance;

const valueInstance = keywordInstance._fromNode(node.parent.children[1], keywordInstance.pointer);
probablyArth marked this conversation as resolved.
Show resolved Hide resolved
if (typeof valueInstance.value() !== "string") {
return;
}

if (!references.has(textDocument.uri)) {
references.set(textDocument.uri, new Map());
}
references.get(textDocument.uri).set(pointer, valueInstance);
};

/**
* @param {import("vscode-languageserver").TextDocuments} documents
* @param {import("vscode-languageserver-textdocument").TextDocument} textDocument
* @param {string} ref
*/
export const validateReference = async (documents, textDocument, ref) => {
const referenceData = await getReferenceData(documents, textDocument.uri, getReferenceForDocument(textDocument.uri).get(ref).value());
if (typeof referenceData === "boolean") {
return referenceData;
}
const { jsonInstance, anchorFragment, localJsonPointer } = referenceData;
if (anchorFragment && (
!isAnchor(anchorFragment) || !await searchAnchorFragment(jsonInstance.textDocument, anchorFragment)
)
) {
return false;
}
if (localJsonPointer && jsonInstance.get(localJsonPointer).value() === undefined) {
return false;
}
return true;
};

/**
* @param {import("vscode-languageserver").TextDocuments} documents
* @param {string} uri
* @param {string} ref
* @returns {Promise<boolean | {
* jsonInstance: JsoncInstance;
* anchorFragment?: string;
* localJsonPointer?: string;
* }>}
*/
const getReferenceData = async (documents, uri, ref) => {
const [$id, fragment] = ref.startsWith("#") ? ref.slice(1).split("#") : ref.split("#");
const embeddedSchema = getEmbeddedIdentifer(uri, $id);

if (ref.startsWith("#")) {
const textDocument = await fetchDocument(documents, uri);
const jsonInstance = JsoncInstance.fromTextDocument(textDocument);
return extractPointers($id, jsonInstance);
}
if (embeddedSchema) {
if (!fragment) {
return true;
}
const jsonInstance = embeddedSchema;
return extractPointers(fragment, jsonInstance);
}
if (!workspaceUri) {
return false;
}
if (isAbsoluteUrl($id)) {
const documentUri = getIdentifier($id);
if (!documentUri) {
return false;
}
if (!fragment) {
return true;
}
const textDocument = await fetchDocument(documents, documentUri);
const jsonInstance = JsoncInstance.fromTextDocument(textDocument);
return extractPointers(fragment, jsonInstance);
}


const currDocument = await fetchDocument(documents, uri);
const schemaResources = await getSchemaResources(currDocument);
const { dialectUri } = schemaResources[0];
const idKeywordName = keywordNameFor("https://json-schema.org/keyword/id", dialectUri);
const jsonInstance = JsoncInstance.fromTextDocument(currDocument);

let fullReferenceUri;
const baseUri = jsonInstance.get(`#/${idKeywordName}`).value();
if (baseUri !== undefined) {
fullReferenceUri = resolve(baseUri, $id);
} else {
fullReferenceUri = pathToFileURL(join(dirname(uri), $id)).toString();
}
const referenceDocumentUri = getIdentifier(fullReferenceUri);
if (!referenceDocumentUri) {
return false;
}
const referenceDocument = await fetchDocument(documents, referenceDocumentUri);
const referenceInstance = JsoncInstance.fromTextDocument(referenceDocument);
if (!fragment) {
return true;
}
return extractPointers(fragment, referenceInstance);
};

const extractPointers = (fragment, jsonInstance) => {
if (!fragment.startsWith("/")) {
const anchorFragment = fragment;
return { jsonInstance, anchorFragment };
} else {
const localJsonPointer = `#${fragment}`;
return { jsonInstance, localJsonPointer };
}
};

/**
*
* @param {TextDocument} textDocument
* @param {string} anchorValue
*/
const searchAnchorFragment = async (textDocument, anchorValue) => {
const schemaResources = await getSchemaResources(textDocument);
for (const { dialectUri } of schemaResources) {
const anchorKeywordName = keywordNameFor("https://json-schema.org/keyword/anchor", dialectUri);
for (const { keywordInstance } of getSemanticTokens(schemaResources)) {
if (keywordInstance.value() === anchorKeywordName) {
const valueInstance = keywordInstance._fromNode(keywordInstance.node.parent.children[1], keywordInstance.pointer);
if (valueInstance.value() === anchorValue && typeof valueInstance.value() === "string") {
return true;
}
}
}
}
return false;
};
Loading