Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fil/xlsx #249

Merged
merged 7 commits into from
Sep 7, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -379,6 +379,10 @@ Returns a promise to the file loads as a [SQLite database client](https://observ
const db = await FileAttachment("chinook.db").sqlite();
```

<a href="#attachment_xlsx" name="attachment_xlsx">#</a> *attachment*.<b>xlsx</b>() [<>](https://github.com/observablehq/stdlib/blob/master/src/xlsx.js "Source")

Returns a promise to the file loaded as an [ExcelWorkbook](https://observablehq.com/@observablehq/excelworkbook).

<a href="#FileAttachments" name="FileAttachments">#</a> <b>FileAttachments</b>(<i>resolve</i>) [<>](https://github.com/observablehq/stdlib/blob/master/src/fileAttachment.js "Source")

*Note: this function is not part of the Observable standard library (in notebooks), but is provided by this module as a means for defining custom file attachment implementations when working directly with the Observable runtime.*
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
"url": "https://github.com/observablehq/stdlib.git"
},
"scripts": {
"test": "tap 'test/**/*-test.js'",
"test": "tap 'test/**/*-test.js' --reporter classic",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was the issue with the default reporter?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exactly the same as tapjs/tapjs#624

~/Sites/observablehq/stdlib fil/xlsx* 41s
❯ yarn test
yarn run v1.22.11
$ tap 'test/**/*-test.js'
Error: Cannot find module './reports/base'
Require stack:
- /Users/fil/Sites/observablehq/stdlib/node_modules/jackspeak/noop.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:927:15)
    at resolveFileName (/Users/fil/Sites/observablehq/stdlib/node_modules/tap/node_modules/resolve-from/index.js:17:39)
    at resolveFrom (/Users/fil/Sites/observablehq/stdlib/node_modules/tap/node_modules/resolve-from/index.js:31:9)
    at module.exports (/Users/fil/Sites/observablehq/stdlib/node_modules/tap/node_modules/resolve-from/index.js:34:41)
    at importJSX (/Users/fil/Sites/observablehq/stdlib/node_modules/tap/node_modules/import-jsx/index.js:24:21)
    at module.exports (/Users/fil/Sites/observablehq/stdlib/node_modules/tap/node_modules/treport/lib/index.js:13:18)
    at exports.makeReporter (/Users/fil/Sites/observablehq/stdlib/node_modules/tap/bin/run.js:422:23)
    at runTests (/Users/fil/Sites/observablehq/stdlib/node_modules/tap/bin/run.js:744:3)
    at mainAsync (/Users/fil/Sites/observablehq/stdlib/node_modules/tap/bin/run.js:244:5)
    at main (/Users/fil/Sites/observablehq/stdlib/node_modules/tap/bin/run.js:127:3) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/Users/fil/Sites/observablehq/stdlib/node_modules/jackspeak/noop.js'
  ]
}
TAP version 13
1..0
# time=0.973ms
----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |        0 |        0 |        0 |        0 |                   |
----------|----------|----------|----------|----------|-------------------|
error Command failed with exit code 1.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh weird. I wasn't running into that.

Copy link
Member

@visnup visnup Sep 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just pushed a change to use the base reporter since it's the default, but with the explicit --reporter flag. let me know if that still works for you.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, unfortunately:

~/Sites/observablehq/stdlib visnup/xlsx*
❯ yarn test
yarn run v1.22.11
$ tap 'test/**/*-test.js' --reporter base
Error: Cannot find module './reports/base'
Require stack:
- /Users/fil/Sites/observablehq/stdlib/node_modules/jackspeak/noop.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:927:15)
    at resolveFileName (/Users/fil/Sites/observablehq/stdlib/node_modules/tap/node_modules/resolve-from/index.js:17:39)
    at resolveFrom (/Users/fil/Sites/observablehq/stdlib/node_modules/tap/node_modules/resolve-from/index.js:31:9)
    at module.exports (/Users/fil/Sites/observablehq/stdlib/node_modules/tap/node_modules/resolve-from/index.js:34:41)
    at importJSX (/Users/fil/Sites/observablehq/stdlib/node_modules/tap/node_modules/import-jsx/index.js:24:21)
    at module.exports (/Users/fil/Sites/observablehq/stdlib/node_modules/tap/node_modules/treport/lib/index.js:13:18)
    at exports.makeReporter (/Users/fil/Sites/observablehq/stdlib/node_modules/tap/bin/run.js:422:23)
    at runTests (/Users/fil/Sites/observablehq/stdlib/node_modules/tap/bin/run.js:744:3)
    at mainAsync (/Users/fil/Sites/observablehq/stdlib/node_modules/tap/bin/run.js:244:5)
    at main (/Users/fil/Sites/observablehq/stdlib/node_modules/tap/bin/run.js:127:3) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/Users/fil/Sites/observablehq/stdlib/node_modules/jackspeak/noop.js'
  ]
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found another reference to the issue: nodejs/citgm#852 (comment)

❯ node -v
v16.9.0

"prepublishOnly": "rollup -c",
"postpublish": "git push && git push --tags"
},
Expand Down
34 changes: 16 additions & 18 deletions src/xlsx.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,28 +6,24 @@ export class ExcelWorkbook {
return this._.worksheets.map((sheet) => sheet.name);
}
sheet(name, {range, headers = false} = {}) {
const sheet = this._.getWorksheet(
typeof name === "number" ? this.sheetNames()[name] : name + ""
);
if (!sheet) throw new Error(`Sheet not found: ${name}`);
const names = this.sheetNames();
const sname = typeof name === "number" ? names[name] : names.includes(name + "") ? name + "" : null;
if (sname == null) throw new Error(`Sheet not found: ${name}`);
const sheet = this._.getWorksheet(sname);
return extract(sheet, {range, headers});
}
}

function extract(sheet, {range, headers}) {
let [[c0, r0], [c1, r1]] = parseRange(range, sheet);
const seen = new Set();
const names = [];
const headerRow = headers && sheet._rows[r0++];
function name(n) {
if (!names[n]) {
let name = (headerRow ? valueOf(headerRow._cells[n]) : AA(n)) || AA(n);
while (seen.has(name)) name += "_";
seen.add((names[n] = name));
}
return names[n];
let names = new Set();
for (let n = c0; n <= c1; n++) {
let name = (headerRow ? valueOf(headerRow._cells[n]) : null) || AA(n);
while (names.has(name)) name += "_";
names.add(name);
}
if (headerRow) for (let c = c0; c <= c1; c++) name(c);
names = new Array(c0).concat(Array.from(names));

const output = new Array(r1 - r0 + 1).fill({});
for (let r = r0; r <= r1; r++) {
Expand All @@ -36,7 +32,7 @@ function extract(sheet, {range, headers}) {
const row = (output[r - r0] = {});
for (let c = c0; c <= c1; c++) {
const value = valueOf(_row._cells[c]);
if (value !== null && value !== undefined) row[name(c)] = value;
if (value != null) row[names[c]] = value;
}
}

Expand Down Expand Up @@ -75,6 +71,8 @@ function parseRange(specifier = [], {columnCount, rowCount}) {
[c0, r0],
[c1, r1],
];
} else {
throw new Error(`Unknown range specifier`);
}
}

Expand All @@ -87,13 +85,13 @@ function AA(c) {
return sc;
}

function NN(s = "") {
const [, sc, sr] = s.match(/^([a-zA-Z]+)?(\d+)?$/);
function NN(s) {
const [, sc, sr] = s.match(/^([A-Z]*)(\d*)$/i);
let c = undefined;
if (sc) {
c = 0;
for (let i = 0; i < sc.length; i++)
c += Math.pow(26, sc.length - i - 1) * (sc.charCodeAt(i) - 64);
}
return [c && c - 1, sr && +sr - 1];
return [c ? c - 1 : undefined, sr ? +sr - 1 : undefined];
}
53 changes: 51 additions & 2 deletions test/xlsx-test.js
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,12 @@ test("FileAttachment.xlsx reads sheet names", (t) => {
t.end();
});

test("FileAttachment.xlsx sheet(name) throws on unknown sheet name", (t) => {
const workbook = new ExcelWorkbook(mockWorkbook({Sheet1: []}));
t.throws(() => workbook.sheet("bad"));
t.end();
});

test("FileAttachment.xlsx reads sheets", (t) => {
const workbook = new ExcelWorkbook(
mockWorkbook({
Expand All @@ -37,25 +43,31 @@ test("FileAttachment.xlsx reads sheets", (t) => {
{A: "one", B: "two", C: "three"},
{A: 1, B: 2, C: 3},
]);
t.same(workbook.sheet("Sheet1"), [
{A: "one", B: "two", C: "three"},
{A: 1, B: 2, C: 3},
]);
t.end();
});

test("FileAttachment.xlsx reads sheets with different types", (t) => {
const workbook = new ExcelWorkbook(
mockWorkbook({
Sheet1: [
["one", {richText: [{text: "two"}, {text: "three"}]}],
["one", null, {richText: [{text: "two"}, {text: "three"}]}, undefined],
[
{text: "link", hyperlink: "https://example.com"},
2,
{formula: "=B2*5", result: 10},
],
[],
],
})
);
t.same(workbook.sheet(0), [
{A: "one", B: "twothree"},
{A: "one", C: "twothree"},
{A: `<a href="https://example.com">link</a>`, B: 2, C: 10},
{},
]);
t.end();
});
Expand Down Expand Up @@ -152,5 +164,42 @@ test("FileAttachment.xlsx reads sheet ranges", (t) => {
t.same(workbook.sheet(0, {range: "2"}), entireSheet.slice(1));
t.same(workbook.sheet(0, {range: [[undefined, 1]]}), entireSheet.slice(1));

// ":I"
// [,[1,]]
const sheetJ = [
{ I: 8, J: 9 },
{ I: 18, J: 19 },
{ I: 28, J: 29 },
{ I: 38, J: 39 }
];
t.same(workbook.sheet(0, {range: "I"}), sheetJ);
t.same(workbook.sheet(0, {range: [[8, undefined], undefined]}), sheetJ);
t.end();
});

test("FileAttachment.xlsx throws on unknown range specifier", (t) => {
const workbook = new ExcelWorkbook(mockWorkbook({ Sheet1: [] }));
t.throws(() => workbook.sheet(0, {range: 0}));
t.end();
});

test("FileAttachment.xlsx derives column names such as A AA AAA…", (t) => {
const l0 = 26 * 26 * 26 + 26 * 26 + 26;
const workbook = new ExcelWorkbook(
mockWorkbook({
Sheet1: [
Array.from({length: l0}).fill(1),
],
})
);
t.same(workbook.sheet(0, {headers: false}).columns.filter(d => d.match(/^A*$/)), ["A", "AA", "AAA"]);
const workbook1 = new ExcelWorkbook(
mockWorkbook({
Sheet1: [
Array.from({length: l0 + 1}).fill(1),
],
})
);
t.same(workbook1.sheet(0, {headers: false}).columns.filter(d => d.match(/^A*$/)), ["A", "AA", "AAA", "AAAA"]);
t.end();
});