Skip to content

Commit

Permalink
improve openspout compat / refactor zip utils / phpstan v2
Browse files Browse the repository at this point in the history
  • Loading branch information
lekoala committed Nov 29, 2024
1 parent 6da073a commit 9dd2326
Show file tree
Hide file tree
Showing 16 changed files with 339 additions and 120 deletions.
46 changes: 7 additions & 39 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ Ideally, importing single sheets of csv or excel should be just a matter of chan

## Supported packages

Native php: very fast csv import/export, but limited features. Can read/output streams.

OpenSpout: fast csv and excel import/export
https://github.com/openspout/openspout

Expand All @@ -18,8 +20,6 @@ https://github.com/thephpleague/csv
PhpSpreadsheet: slow excel (xls and xlsx) and csv import/export, but more features
https://github.com/PHPOffice/PhpSpreadsheet

Native php: very fast csv import/export, but limited features. Can read/output streams.

SimpleXLSX: very fast excel import/export
https://github.com/shuchkin/simplexlsx
https://github.com/shuchkin/simplexlsxgen
Expand Down Expand Up @@ -100,45 +100,13 @@ This package supports only 1 worksheet, as it is meant to be able to replace csv

## Benchmarks

Since we can compare our solutions, there is a built in bench.php script that give the following results on my machine
These results are run with:

openspout/openspout 4.24.4
phpoffice/phpspreadsheet 2.1.0
league/csv 9.16.0
shuchkin/simplexlsx 1.1.11
shuchkin/simplexlsxgen 1.4.11

Reading a file with 5000 rows:

Results for csv
LeKoala\SpreadCompat\Csv\Native : 0.0084
LeKoala\SpreadCompat\Csv\League : 0.0317
LeKoala\SpreadCompat\Csv\OpenSpout : 0.0993
LeKoala\SpreadCompat\Csv\PhpSpreadsheet : 0.6713

Results for xlsx
LeKoala\SpreadCompat\Xlsx\Native : 0.0639
LeKoala\SpreadCompat\Xlsx\Simple : 0.1476
LeKoala\SpreadCompat\Xlsx\PhpSpreadsheet : 0.7436
LeKoala\SpreadCompat\Xlsx\OpenSpout : 0.8979


Write a file with 2500 rows:
Since we can compare our solutions, there is a built in bench script. You can check the results here

Results for csv
LeKoala\SpreadCompat\Csv\Native : 0.0065
LeKoala\SpreadCompat\Csv\League : 0.0129
LeKoala\SpreadCompat\Csv\OpenSpout : 0.0443
LeKoala\SpreadCompat\Csv\PhpSpreadsheet : 0.3668
- [read benchmark](docs/bench-read.md)
- [write benchmark](docs/bench-write.md)

Results for xlsx
LeKoala\SpreadCompat\Xlsx\Native : 0.0401
LeKoala\SpreadCompat\Xlsx\Simple : 0.0861
LeKoala\SpreadCompat\Xlsx\OpenSpout : 0.2555
LeKoala\SpreadCompat\Xlsx\PhpSpreadsheet : 0.674
For simple imports/exports, it's very clear that using the `Native` adapter is the fastest.

For simple imports/exports, it's very clear that using the Native adapter is the fastest.
These are not enabled by default since they might lack some compatibility feature you may require.
Otherwise, `league/csv` and `shuchkin/simplexlsx` are great choices.

Stop wasting cpu cycles right now and please use the most efficient adapter :-)
8 changes: 4 additions & 4 deletions bench-read.php → bin/bench-read.php
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@
use LeKoala\SpreadCompat\Xlsx\OpenSpout as XlsxOpenSpout;
use LeKoala\SpreadCompat\Xlsx\Simple;

require './vendor/autoload.php';
require dirname(__DIR__) . '/vendor/autoload.php';

$largeCsv = __DIR__ . '/tests/data/large.csv';
$largeXlsx = __DIR__ . '/tests/data/large.xlsx';
$largeCsv = dirname(__DIR__) . '/tests/data/large.csv';
$largeXlsx = dirname(__DIR__) . '/tests/data/large.xlsx';

$csv = [
League::class,
Expand Down Expand Up @@ -67,7 +67,7 @@
$results[$class] = $averageTime;
}

uasort($results, fn ($a, $b) => $a <=> $b);
uasort($results, fn($a, $b) => $a <=> $b);
foreach ($results as $class => $averageTime) {
echo "$class : " . $averageTime . PHP_EOL;
}
Expand Down
4 changes: 2 additions & 2 deletions bench-write.php → bin/bench-write.php
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
use LeKoala\SpreadCompat\Xlsx\OpenSpout as XlsxOpenSpout;
use LeKoala\SpreadCompat\Xlsx\Simple;

require './vendor/autoload.php';
require dirname(__DIR__) . '/vendor/autoload.php';

$largeCsv = SpreadCompat::getTempFilename();
$largeXlsx = SpreadCompat::getTempFilename();
Expand Down Expand Up @@ -75,7 +75,7 @@
$results[$class] = $averageTime;
}

uasort($results, fn ($a, $b) => $a <=> $b);
uasort($results, fn($a, $b) => $a <=> $b);
foreach ($results as $class => $averageTime) {
echo "$class : " . $averageTime . PHP_EOL;
}
Expand Down
10 changes: 7 additions & 3 deletions composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@
"maennchen/zipstream-php": "^3.1",
"openspout/openspout": "^4",
"phpoffice/phpspreadsheet": "^1.26|^2",
"phpstan/phpstan": "^1",
"phpunit/phpunit": "^10",
"phpstan/phpstan": "^2",
"phpunit/phpunit": "^10|^11",
"shuchkin/simplexlsx": "^1",
"shuchkin/simplexlsxgen": "^1.3",
"squizlabs/php_codesniffer": "^3.6"
Expand Down Expand Up @@ -58,6 +58,10 @@
"phpunit-only": "phpunit --group=only",
"phpcs": "phpcs",
"phpstan": "phpstan analyse src/ --memory-limit=-1",
"serve": "php -S localhost:8001 -t ./"
"serve": "php -S localhost:8001 -t ./",
"bench": [
"php ./bin/bench-read.php > ./docs/bench-read.md",
"php ./bin/bench-write.php > ./docs/bench-write.md"
]
}
}
12 changes: 12 additions & 0 deletions docs/bench-read.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Results for csv
LeKoala\SpreadCompat\Csv\Native : 0.008
LeKoala\SpreadCompat\Csv\League : 0.0312
LeKoala\SpreadCompat\Csv\OpenSpout : 0.0979
LeKoala\SpreadCompat\Csv\PhpSpreadsheet : 1.3744

Results for xlsx
LeKoala\SpreadCompat\Xlsx\Native : 0.0617
LeKoala\SpreadCompat\Xlsx\Simple : 0.1396
LeKoala\SpreadCompat\Xlsx\OpenSpout : 0.9332
LeKoala\SpreadCompat\Xlsx\PhpSpreadsheet : 1.4632

12 changes: 12 additions & 0 deletions docs/bench-write.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Results for csv
LeKoala\SpreadCompat\Csv\Native : 0.0074
LeKoala\SpreadCompat\Csv\League : 0.0125
LeKoala\SpreadCompat\Csv\OpenSpout : 0.0433
LeKoala\SpreadCompat\Csv\PhpSpreadsheet : 0.7075

Results for xlsx
LeKoala\SpreadCompat\Xlsx\Native : 0.0402
LeKoala\SpreadCompat\Xlsx\Simple : 0.0845
LeKoala\SpreadCompat\Xlsx\OpenSpout : 0.2622
LeKoala\SpreadCompat\Xlsx\PhpSpreadsheet : 1.017

3 changes: 2 additions & 1 deletion phpstan.neon.dist
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,13 @@ parameters:
- src
reportUnmatchedIgnoredErrors: true
ignoreErrors:
- "#(.*)return type has no value type specified in iterable type array#"
- "#(.*)with no value type specified in iterable type iterable#"
- "#(.*)has parameter \\$opts with no type specified.#"
- "#Parameter (.*) \\$opts#"
- "#(.*)array_combine expects(.*)#"
- "#(.*)SpreadInterface but returns object.#"
- "#but does not specify its types: TValue#"
- "#method_exists(.*) OpenSpout#"
# https://backendtea.com/post/use-phpstan-bleeding-edge/
includes:
- vendor/phpstan/phpstan/conf/bleedingEdge.neon
38 changes: 38 additions & 0 deletions src/Common/ZipUtils.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
<?php

declare(strict_types=1);

namespace LeKoala\SpreadCompat\Common;

use ZipArchive;

class ZipUtils
{
public static function zipError(int $code): string
{
return match ($code) {
ZipArchive::ER_EXISTS => 'File already exists.',
ZipArchive::ER_INCONS => 'Zip archive inconsistent.',
ZipArchive::ER_INVAL => 'Invalid argument.',
ZipArchive::ER_MEMORY => 'Malloc failure.',
ZipArchive::ER_NOENT => 'No such file.',
ZipArchive::ER_NOZIP => 'Not a zip archive.',
ZipArchive::ER_OPEN => 'Can\'t open file.',
ZipArchive::ER_READ => 'Read error.',
ZipArchive::ER_SEEK => 'Seek error.',
default => 'Unknown error code ' . $code . '.',
};
}

public static function getData(ZipArchive $zip, string $name): ?string
{
$idx = $zip->locateName($name);
if ($idx) {
$result = $zip->getFromIndex($idx);
if ($result) {
return $result;
}
}
return null;
}
}
26 changes: 24 additions & 2 deletions src/Csv/League.php
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,11 @@ public function readFile(string $filename, ...$opts): Generator
return $this->read(Reader::createFromPath($filename));
}

/**
* @param iterable<array<float|int|string|\Stringable|null>> $data
* @param mixed ...$opts
* @return string
*/
public function writeString(iterable $data, ...$opts): string
{
$this->configure(...$opts);
Expand All @@ -50,6 +55,12 @@ public function writeString(iterable $data, ...$opts): string
return $csv->toString();
}

/**
* @param iterable<array<float|int|string|\Stringable|null>> $data
* @param string $filename
* @param mixed ...$opts
* @return bool
*/
public function writeFile(iterable $data, string $filename, ...$opts): bool
{
try {
Expand All @@ -62,6 +73,12 @@ public function writeFile(iterable $data, string $filename, ...$opts): bool
}
}

/**
* @param iterable<array<float|int|string|\Stringable|null>> $data
* @param string $filename
* @param mixed ...$opts
* @return void
*/
public function output(iterable $data, string $filename, ...$opts): void
{
$this->configure(...$opts);
Expand All @@ -83,6 +100,11 @@ protected function read(Reader $csv): Generator
}
}

/**
* @param iterable<array<float|int|string|\Stringable|null>> $data
* @param Writer $csv
* @return void
*/
protected function write(iterable $data, Writer $csv): void
{
$this->initialize($csv);
Expand Down Expand Up @@ -127,8 +149,8 @@ protected function initialize(Reader|Writer $csv): void

$csv->addFormatter(
(new CharsetConverter())
->inputEncoding($inputEncoding)
->outputEncoding($outputEncoding)
->inputEncoding($inputEncoding)
->outputEncoding($outputEncoding)
);
}
}
19 changes: 18 additions & 1 deletion src/Csv/Native.php
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ public function readFile(

/**
* @param resource $stream
* @param iterable $data
* @param iterable<array<float|int|string|\Stringable|null>> $data
* @return void
*/
protected function write($stream, iterable $data): void
Expand All @@ -88,6 +88,11 @@ protected function write($stream, iterable $data): void
}
}

/**
* @param iterable<array<float|int|string|\Stringable|null>> $data
* @param mixed ...$opts
* @return string
*/
public function writeString(
iterable $data,
...$opts
Expand All @@ -101,6 +106,12 @@ public function writeString(
return $contents;
}

/**
* @param iterable<array<float|int|string|\Stringable|null>> $data
* @param string $filename
* @param mixed ...$opts
* @return bool
*/
public function writeFile(
iterable $data,
string $filename,
Expand All @@ -113,6 +124,12 @@ public function writeFile(
return true;
}

/**
* @param iterable<array<float|int|string|\Stringable|null>> $data
* @param string $filename
* @param mixed ...$opts
* @return void
*/
public function output(
iterable $data,
string $filename,
Expand Down
17 changes: 17 additions & 0 deletions src/Csv/OpenSpout.php
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,11 @@ protected function getWriter(): Writer
return $writer;
}

/**
* @param iterable<list<bool|\DateInterval|\DateTimeInterface|float|int|string|null>> $data
* @param mixed ...$opts
* @return string
*/
public function writeString(iterable $data, ...$opts): string
{
$this->configure(...$opts);
Expand All @@ -89,6 +94,12 @@ public function writeString(iterable $data, ...$opts): string
return $contents;
}

/**
* @param iterable<list<bool|\DateInterval|\DateTimeInterface|float|int|string|null>> $data
* @param string $filename
* @param mixed ...$opts
* @return bool
*/
public function writeFile(iterable $data, string $filename, ...$opts): bool
{
$this->configure(...$opts);
Expand All @@ -104,6 +115,12 @@ public function writeFile(iterable $data, string $filename, ...$opts): bool
return true;
}

/**
* @param iterable<list<bool|\DateInterval|\DateTimeInterface|float|int|string|null>> $data
* @param string $filename
* @param mixed ...$opts
* @return void
*/
public function output(iterable $data, string $filename, ...$opts): void
{
$this->configure(...$opts);
Expand Down
Loading

0 comments on commit 9dd2326

Please sign in to comment.